-- Logs begin at Mon 2022-05-16 14:13:51 UTC, end at Mon 2022-05-16 15:43:16 UTC. -- May 16 14:54:38.423254 dv-cinder-841753-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 16 14:54:38.432055 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 16 14:54:38 2022] *** May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: nodename: dv-cinder-841753-2-iscsi May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: machine: x86_64 May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: clock source: unix May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: pcre jit disabled May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: detected number of CPU cores: 4 May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: current working directory: / May 16 14:54:38.434051 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: detected binary path: /usr/bin/uwsgi-core May 16 14:54:38.434610 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: your processes number limit is 47680 May 16 14:54:38.434610 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: your memory page size is 4096 bytes May 16 14:54:38.434610 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: detected max file descriptor number: 2048 May 16 14:54:38.434610 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: lock engine: pthread robust mutexes May 16 14:54:38.434610 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: thunder lock: enabled May 16 14:54:38.434610 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 16 14:54:38.434610 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 16 14:54:38.480047 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: Python main interpreter initialized at 0x5599c87a79c0 May 16 14:54:38.480047 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: python threads support enabled May 16 14:54:38.480047 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: your server socket listen backlog is limited to 100 connections May 16 14:54:38.480047 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: your mercy for graceful operations on workers is 90 seconds May 16 14:54:38.480409 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: mapped 403077 bytes (393 KB) for 2 cores May 16 14:54:38.480486 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: *** Operational MODE: preforking *** May 16 14:54:38.480737 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: *** uWSGI is running in multiple interpreter mode *** May 16 14:54:38.480737 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: spawned uWSGI master process (pid: 80221) May 16 14:54:38.480937 dv-cinder-841753-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 16 14:54:38.487655 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: spawned uWSGI worker 1 (pid: 80232, cores: 1) May 16 14:54:38.487655 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: spawned uWSGI worker 2 (pid: 80233, cores: 1) May 16 14:54:38.487655 dv-cinder-841753-2-iscsi devstack@keystone.service[80221]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 16 14:54:44.227056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 14:54:44.227056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes May 16 14:54:44.228100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 14:54:44.228100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: from cryptography.utils import int_from_bytes May 16 14:54:44.228548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 14:54:44.228548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes May 16 14:54:44.229411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 14:54:44.229411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: from cryptography.utils import int_from_bytes May 16 14:54:44.277284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 16 14:54:44.277474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 16 14:54:44.277664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] command line args: [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 16 14:54:44.277929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 16 14:54:44.278105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ================================================================================ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 16 14:54:44.278214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 16 14:54:44.278355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.278439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 16 14:54:44.278592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] command line args: [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 16 14:54:44.278676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_dir = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.278777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.278859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 16 14:54:44.278958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_source = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.279035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ================================================================================ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 16 14:54:44.279135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] debug = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.279269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] admin_token = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.279352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.279621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config_dir = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.279768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.279866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.279965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config_source = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] debug = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_config_append = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.280965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_dir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_config_append = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.281919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_dir = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.282978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.283964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.284986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.285105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.285227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] publish_errors = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.285382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.285499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.285616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.285775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.285889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] publish_errors = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] standard_threads = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.286992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.287124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.287124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.287400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.287540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_journal = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.287678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] standard_threads = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.287806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_json = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.287806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_stderr = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.288020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.288130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_syslog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.288271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.288396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.288545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.288545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_journal = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.288808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.289029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_json = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.289150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.289262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_stderr = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.289262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_syslog = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.289483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.289595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 14:54:44.289713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.289841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.289993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.290917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.291970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.292084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.292273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.292388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.292501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.292655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.292775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.292893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293484 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.293952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.294988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.295199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.295310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.295421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.295531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.295669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.295791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.295903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.296923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.297998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.298923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.299978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.300990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301682 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.301927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.302978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.303094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.303214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.303327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.303449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.303563 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.303692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.303806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.304525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.304779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.304879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.304973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.305904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.306925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.307902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.308930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.309050 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.309164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.309282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.309395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.309508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.309904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.310940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.311996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.312921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.external = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.313962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314407 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314407 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.password = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.314861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.token = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.external = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.315944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.password = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.token = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.316975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.317979 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.318928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.319043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.319156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.319277 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.319393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.320959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.321835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.322983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.323916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.324958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325869 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.325980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.326899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.327835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.328919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329042 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.329978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.330889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.331947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332858 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.332972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.333093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.333207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.333320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.333432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.333796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.333908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.334953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.335910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.336961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.337881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.338941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339748 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.339975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.340963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.341977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.342892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.343992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.344903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345477 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.345947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346635 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.346974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.347914 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.348929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.349948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.350898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.351909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.352919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.353950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.354978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.355924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.356956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.357976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.358992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.359946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.360923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.361903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.362926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 16 14:54:44.363390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.363390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.364165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 14:54:44.364165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 16 14:54:44.393364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.393589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.393811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.394370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.394601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.394828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.403205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.404175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.431810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.432037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.432996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.433231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.485894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.486105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.537637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.537809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.537897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.537974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.604633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.604859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.605904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.606117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.628661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.628892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.628983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.629964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.630055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.630228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.641561 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.641851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.656609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.656858 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.657904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.658045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.658150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.658311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 14:54:44.675377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.675582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 14:54:44.675780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.675940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.676022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 14:54:44.676143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 14:54:44.676310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.676400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.676511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 14:54:44.676649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.676778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.676903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.677037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.677153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.677544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.688159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.688434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.688562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.688698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.688840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.688975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.689109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.689389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.697712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.698044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.701005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.701284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.701367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.701563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.701660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.701877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.701959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.702209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.702291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.702465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.702549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.702729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.702812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.702995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.703077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.703258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.703340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.703522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.703603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.703810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.703914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.704084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.731580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.732085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.733204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.733480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.733694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.733971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.738757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.738991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.739242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.739335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.739518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.739838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.747109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.747385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.747745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.748027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.752006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.752290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.752567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.752732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.752896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.753969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.754206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.754471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.772696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.772978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.773452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.773727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.777546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.778307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.781207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.781501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.781968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.782264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.786553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.787272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.790901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.791445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.794633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.794927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.795204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.795287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.795499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.795594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.795796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.795892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.796075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.796167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.796357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.796638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.815762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.816136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.816308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.816393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.816649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.816730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.816944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.817205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.824867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.825134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.826576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.826849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.826933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.827119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.829897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.830173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.830255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.830449 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.830541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.830731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.830813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.831002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.831084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.831281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.844848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.845081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.845168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.845364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.845470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.845641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.845725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.845913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.845992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.846188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.846269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.846458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.846537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.846734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.864954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.865448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.869470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.869581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.872800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.872921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.874395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.874491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.876038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.876221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.876350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.876490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.876624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.876757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.884013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.884172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.885602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.885736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.888843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.888984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.889140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.889275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.889416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.889549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.889689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.889844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.899001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.899090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.899300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.899386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.899578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.899709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.907483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.907569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 14:54:44.907780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.907864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.908957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.909139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.909221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.909408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.909498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.909671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.909752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.909930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.910976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.911054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 14:54:44.950852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5599c87a79c0 pid: 80233 (default app) May 16 14:54:44.950965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5599c87a79c0 pid: 80232 (default app) May 16 14:54:44.954198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207b9e-7d07-4350-a5eb-766c3de780fd None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:44.954487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207b9e-7d07-4350-a5eb-766c3de780fd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:44.954768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207b9e-7d07-4350-a5eb-766c3de780fd None None] PATH_INFO: `/v3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:44.959200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1/1] 10.222.0.71 () {50 vars in 791 bytes} [Mon May 16 14:54:44 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:54:53.764075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433aa52-3c67-404b-bf0d-da6ef6b49716 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:53.764662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433aa52-3c67-404b-bf0d-da6ef6b49716 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:53.764662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433aa52-3c67-404b-bf0d-da6ef6b49716 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:53.769015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1/2] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:54:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:54:53.774221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:53.774721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:53.774826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:53.806310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG oslo_db.sqlalchemy.engines [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 16 14:54:53.991524 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80233) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 16 14:54:53.992560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80233) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 16 14:54:54.047396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:54:54.048950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:54:54.170966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 2/3] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:54:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 16 14:54:54.336536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433aa52-3c67-404b-bf0d-da6ef6b49716 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:54.370074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG oslo_db.sqlalchemy.engines [None req-b433aa52-3c67-404b-bf0d-da6ef6b49716 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80232) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 16 14:54:54.496421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433aa52-3c67-404b-bf0d-da6ef6b49716 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:54.500587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be7710-76f4-45ea-879d-048760e3bceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:54.501230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be7710-76f4-45ea-879d-048760e3bceb admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:54.501508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be7710-76f4-45ea-879d-048760e3bceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:54.501777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be7710-76f4-45ea-879d-048760e3bceb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:54.510263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be7710-76f4-45ea-879d-048760e3bceb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:54:54.561372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING keystone.common.rbac_enforcer.enforcer [None req-84be7710-76f4-45ea-879d-048760e3bceb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 16 14:54:54.561706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be7710-76f4-45ea-879d-048760e3bceb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:54:54.570954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 2/4] 10.222.0.71 () {58 vars in 1324 bytes} [Mon May 16 14:54:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 16 14:54:54.577475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:54.592083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e49d0-2c0f-4639-8eac-1471b332776c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:54.596466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7fa3c-782c-4008-b191-b51257f4e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:54.597588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7fa3c-782c-4008-b191-b51257f4e852 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:54.597876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7fa3c-782c-4008-b191-b51257f4e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:54.598155 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7fa3c-782c-4008-b191-b51257f4e852 admin admin] PATH_INFO: `/v3/projects/63e6523cee6045dabb771a1e00b70243` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:54.599707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7fa3c-782c-4008-b191-b51257f4e852 admin admin] RBAC: Authorizing `identity:get_project(project_id=63e6523cee6045dabb771a1e00b70243)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:54:54.649644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING keystone.common.rbac_enforcer.enforcer [None req-39a7fa3c-782c-4008-b191-b51257f4e852 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 16 14:54:54.649979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7fa3c-782c-4008-b191-b51257f4e852 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:54:54.654491 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 3/5] 10.222.0.71 () {56 vars in 1207 bytes} [Mon May 16 14:54:54 2022] GET /identity/v3/projects/63e6523cee6045dabb771a1e00b70243 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:54:56.855977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bbb4a-ea38-4789-b87c-f5dbf1b6327f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:56.856367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bbb4a-ea38-4789-b87c-f5dbf1b6327f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:56.856534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bbb4a-ea38-4789-b87c-f5dbf1b6327f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:56.860634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 3/6] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:54:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:54:56.865960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d853-c4e1-49c6-91b6-af473f1e0791 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:56.866380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d853-c4e1-49c6-91b6-af473f1e0791 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:56.866519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d853-c4e1-49c6-91b6-af473f1e0791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:56.887097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a017d853-c4e1-49c6-91b6-af473f1e0791 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:54:56.888660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a017d853-c4e1-49c6-91b6-af473f1e0791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:54:56.896563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 4/7] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:54:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 16 14:54:56.919833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bbb4a-ea38-4789-b87c-f5dbf1b6327f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:56.932117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bbb4a-ea38-4789-b87c-f5dbf1b6327f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:56.936288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab6da6-7d2c-43ef-b9ba-b4417111768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:56.936918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab6da6-7d2c-43ef-b9ba-b4417111768c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:56.937202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab6da6-7d2c-43ef-b9ba-b4417111768c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:56.937480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab6da6-7d2c-43ef-b9ba-b4417111768c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:56.944960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab6da6-7d2c-43ef-b9ba-b4417111768c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:54:56.947733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab6da6-7d2c-43ef-b9ba-b4417111768c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:54:56.956247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 4/8] 10.222.0.71 () {58 vars in 1324 bytes} [Mon May 16 14:54:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 14:54:56.962395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017d853-c4e1-49c6-91b6-af473f1e0791 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:56.971264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017d853-c4e1-49c6-91b6-af473f1e0791 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:56.975682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eecda-c653-4231-9712-8298cb858363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:56.976568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eecda-c653-4231-9712-8298cb858363 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:56.976855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eecda-c653-4231-9712-8298cb858363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:56.977134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eecda-c653-4231-9712-8298cb858363 admin admin] PATH_INFO: `/v3/users/6246a26f46e94e7389a45ee846fec1c9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:56.978519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eecda-c653-4231-9712-8298cb858363 admin admin] RBAC: Authorizing `identity:get_user(user_id=6246a26f46e94e7389a45ee846fec1c9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:54:56.980367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eecda-c653-4231-9712-8298cb858363 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:54:56.981921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 5/9] 10.222.0.71 () {56 vars in 1198 bytes} [Mon May 16 14:54:56 2022] GET /identity/v3/users/6246a26f46e94e7389a45ee846fec1c9 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:54:59.262693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db9aec-3956-4105-81a8-d00b08b9eb36 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.263228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db9aec-3956-4105-81a8-d00b08b9eb36 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.263228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db9aec-3956-4105-81a8-d00b08b9eb36 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.264728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 5/10] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:54:59.270191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb9260-eed4-4b23-89c1-e8aa6451ae4d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.270577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb9260-eed4-4b23-89c1-e8aa6451ae4d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.270755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb9260-eed4-4b23-89c1-e8aa6451ae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.296842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d4a8f-6a8b-4f08-b8a6-b56f9a0fef42 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.297126 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d4a8f-6a8b-4f08-b8a6-b56f9a0fef42 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.297402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d4a8f-6a8b-4f08-b8a6-b56f9a0fef42 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.298772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 6/11] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:54:59.305888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.306266 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.306458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.347896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG passlib.handlers.bcrypt [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80232) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 16 14:54:59.348321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG passlib.handlers.bcrypt [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80232) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 16 14:54:59.395004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:54:59.395536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-04fb9260-eed4-4b23-89c1-e8aa6451ae4d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:54:59.396642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:54:59.396747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-04fb9260-eed4-4b23-89c1-e8aa6451ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:54:59.475505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 6/12] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:54:59.493181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 7/13] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:54:59.497386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb9260-eed4-4b23-89c1-e8aa6451ae4d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:59.515529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:59.593358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb9260-eed4-4b23-89c1-e8aa6451ae4d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:59.609924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031270b3-6326-4a5b-bae7-ce63aaec47bf None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:59.625887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:59.626537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.626825 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.627103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.641273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:59.642300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.642582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.642855 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.654520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:54:59.666135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:54:59.667241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:54:59.672954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 8/14] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:54:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:54:59.673625 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:59.692876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:54:59.694365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:54:59.769638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0dea5-2f95-4bdb-8012-142838cbf6b1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:59.788000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 7/15] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:54:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:54:59.793449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:59.801517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:59.802180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.802472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.802750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.803499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:54:59.803840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:54:59.805534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:54:59.896837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4be108-ba0a-473b-ad10-075445876bab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:54:59.924898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 9/16] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:54:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:54:59.931158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:54:59.932560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:54:59.933362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:54:59.933653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:54:59.933930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:54:59.943843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:54:59.945313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:54:59.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:54:59.966772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:54:59.967959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:54:59.967959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:54:59.967959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:54:59.967959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:54:59.967959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 8/17] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:54:59.972930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:00.025060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b1769-11a8-40f6-aba2-4404e159004f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:00.056972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:00.057836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:00.058116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:00.058389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] PATH_INFO: `/v3/users/6246a26f46e94e7389a45ee846fec1c9` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:00.059911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] RBAC: Authorizing `identity:get_user(user_id=6246a26f46e94e7389a45ee846fec1c9)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:00.061347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:00.062950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 10/18] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:54:59 2022] GET /identity/v3/users/6246a26f46e94e7389a45ee846fec1c9 => generated 247 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:00.068277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:00.069265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9feff-856f-44c6-bb25-d20329cdb6c1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:00.101328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:00.102671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:00.102949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:00.103222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:00.103971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:00.104277 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:00.105655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:00.122249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 9/19] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:00.165634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938d51e-0147-4ebd-953e-3b5566a4ce22 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:00.201639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494fb8c-7b42-4303-91c4-d66e40eda216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:00.202448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494fb8c-7b42-4303-91c4-d66e40eda216 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:00.202727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494fb8c-7b42-4303-91c4-d66e40eda216 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:00.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494fb8c-7b42-4303-91c4-d66e40eda216 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:00.204244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494fb8c-7b42-4303-91c4-d66e40eda216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:00.205717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494fb8c-7b42-4303-91c4-d66e40eda216 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:00.207584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 11/20] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:00.214482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:00.315098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e1894-16ee-439b-825d-266f81ec1107 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:00.347902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471e5f9-8d83-404b-a524-1880209e36a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:00.348571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471e5f9-8d83-404b-a524-1880209e36a9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:00.348848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471e5f9-8d83-404b-a524-1880209e36a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:00.349125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471e5f9-8d83-404b-a524-1880209e36a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:00.349953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471e5f9-8d83-404b-a524-1880209e36a9 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.domain.id=default, user.id=6246a26f46e94e7389a45ee846fec1c9) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:00.350262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471e5f9-8d83-404b-a524-1880209e36a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:00.351752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471e5f9-8d83-404b-a524-1880209e36a9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:00.358472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 10/21] 10.222.0.71 () {56 vars in 1322 bytes} [Mon May 16 14:55:00 2022] GET /identity/v3/role_assignments?user.id=6246a26f46e94e7389a45ee846fec1c9&scope.domain.id=default&role.id=1b3a0717a878438ea224dec0c7639471 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:02.787992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85816384-8b10-4be0-8355-22b5077a04a4 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:02.789086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85816384-8b10-4be0-8355-22b5077a04a4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:02.790286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85816384-8b10-4be0-8355-22b5077a04a4 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:02.792520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 12/22] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:02.798900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d95e7d-79d4-4918-a40e-982fe85bf209 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:02.799806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d95e7d-79d4-4918-a40e-982fe85bf209 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:02.800306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d95e7d-79d4-4918-a40e-982fe85bf209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:02.824608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-37d95e7d-79d4-4918-a40e-982fe85bf209 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:02.826134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-37d95e7d-79d4-4918-a40e-982fe85bf209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:02.913286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 11/23] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:02.953469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85816384-8b10-4be0-8355-22b5077a04a4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:03.000030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b5d6-98c4-428c-85a4-613fdd633517 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.000659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b5d6-98c4-428c-85a4-613fdd633517 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.001166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b5d6-98c4-428c-85a4-613fdd633517 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.003080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 12/24] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:03.009222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34881b6b-d3bc-461d-a4bd-a695018f89e0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.009949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34881b6b-d3bc-461d-a4bd-a695018f89e0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.010431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34881b6b-d3bc-461d-a4bd-a695018f89e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.039815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-34881b6b-d3bc-461d-a4bd-a695018f89e0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:03.041316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-34881b6b-d3bc-461d-a4bd-a695018f89e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:03.084716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85816384-8b10-4be0-8355-22b5077a04a4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:03.137290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:03.138098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.138380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.138652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.158210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 13/25] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:03.169835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:03.171333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:03.183449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b69a-6d8d-423b-89cc-2f5339fab544 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.183790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b69a-6d8d-423b-89cc-2f5339fab544 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.184074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b69a-6d8d-423b-89cc-2f5339fab544 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.185593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 14/26] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:03.191369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac94b-7455-4298-8ec5-b38bf10520fa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.191709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac94b-7455-4298-8ec5-b38bf10520fa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.191965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac94b-7455-4298-8ec5-b38bf10520fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:55:03.204387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:03.205205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:03.206005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:55:03.206005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:55:03.206005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:55:03.206005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:03.210062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 13/27] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:03.214820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:03.218777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c2eac94b-7455-4298-8ec5-b38bf10520fa None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:03.219999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2eac94b-7455-4298-8ec5-b38bf10520fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:03.285951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 15/28] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:03.287137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac94b-7455-4298-8ec5-b38bf10520fa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:03.356584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed083f4c-0c74-46bc-8560-d2e08a9463aa None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:03.452603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:03.453677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.454281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.454776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.455787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:03.456343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:03.457997 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:03.486019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 14/29] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:03.487168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:03.512758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac94b-7455-4298-8ec5-b38bf10520fa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:03.575508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:03.576337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.576623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.576900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.597197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:03.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:03.607158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:03.607964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:03.612337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 16/30] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:03.613191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:03.651747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ee09-6c30-480d-99cf-b66dcf9165b5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:03.696318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3d574-bfec-4c75-bb5a-77984409352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:03.697530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3d574-bfec-4c75-bb5a-77984409352e None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.698345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3d574-bfec-4c75-bb5a-77984409352e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.699074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3d574-bfec-4c75-bb5a-77984409352e None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.700052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3d574-bfec-4c75-bb5a-77984409352e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:03.702083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3d574-bfec-4c75-bb5a-77984409352e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:03.716763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe1a3-3798-4677-8024-ab1aa7bf14dc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:03.750475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbb8e8-e4f5-4632-85ff-6596d49c790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:03.751142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb8e8-e4f5-4632-85ff-6596d49c790e None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.751427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb8e8-e4f5-4632-85ff-6596d49c790e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.751736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb8e8-e4f5-4632-85ff-6596d49c790e None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.752542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbb8e8-e4f5-4632-85ff-6596d49c790e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:03.754003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbb8e8-e4f5-4632-85ff-6596d49c790e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:03.901952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 15/31] 10.222.0.71 () {60 vars in 1131 bytes} [Mon May 16 14:55:03 2022] POST /identity/v3/roles => generated 225 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:03.904275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df36fa-af38-4116-a075-76ea6aef9036 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.904801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df36fa-af38-4116-a075-76ea6aef9036 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.905294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df36fa-af38-4116-a075-76ea6aef9036 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:03.907258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 16/32] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:03.908380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df36fa-af38-4116-a075-76ea6aef9036 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:03.974384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 17/33] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:55:03 2022] POST /identity/v3/projects => generated 301 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:03.985079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf863e-78a0-42b9-971c-37b9a6390c29 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:03.985749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf863e-78a0-42b9-971c-37b9a6390c29 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:03.986289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf863e-78a0-42b9-971c-37b9a6390c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.009952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-69bf863e-78a0-42b9-971c-37b9a6390c29 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:04.012077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-69bf863e-78a0-42b9-971c-37b9a6390c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:04.030278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df36fa-af38-4116-a075-76ea6aef9036 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:04.076630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196859dc-44d3-439d-935f-a45547610176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:04.077545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196859dc-44d3-439d-935f-a45547610176 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.077976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196859dc-44d3-439d-935f-a45547610176 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.078394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196859dc-44d3-439d-935f-a45547610176 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.079314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196859dc-44d3-439d-935f-a45547610176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:04.079806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196859dc-44d3-439d-935f-a45547610176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:04.081719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196859dc-44d3-439d-935f-a45547610176 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:04.098498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 17/34] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:04.101265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196859dc-44d3-439d-935f-a45547610176 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:04.137537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 18/35] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:04.165486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf863e-78a0-42b9-971c-37b9a6390c29 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:04.283048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196859dc-44d3-439d-935f-a45547610176 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:04.326703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb07188-7855-4167-bac5-f5aac31d6d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:04.327364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf863e-78a0-42b9-971c-37b9a6390c29 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:04.327585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb07188-7855-4167-bac5-f5aac31d6d63 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.327908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb07188-7855-4167-bac5-f5aac31d6d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.328183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb07188-7855-4167-bac5-f5aac31d6d63 None admin] PATH_INFO: `/v3/users/6246a26f46e94e7389a45ee846fec1c9` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.329745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb07188-7855-4167-bac5-f5aac31d6d63 None admin] RBAC: Authorizing `identity:get_user(user_id=6246a26f46e94e7389a45ee846fec1c9)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:04.332366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb07188-7855-4167-bac5-f5aac31d6d63 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:04.334283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 18/36] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:04 2022] GET /identity/v3/users/6246a26f46e94e7389a45ee846fec1c9 => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:04.335976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fb6f-7eaa-41ac-b21b-b2878525a15e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.337047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fb6f-7eaa-41ac-b21b-b2878525a15e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.338175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fb6f-7eaa-41ac-b21b-b2878525a15e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.340393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 19/37] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:04.342378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83fb6f-7eaa-41ac-b21b-b2878525a15e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:04.373932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce2490-8e30-49de-84f5-d356792b8c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:04.374855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2490-8e30-49de-84f5-d356792b8c95 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.375287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2490-8e30-49de-84f5-d356792b8c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.375574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce2490-8e30-49de-84f5-d356792b8c95 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.376453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce2490-8e30-49de-84f5-d356792b8c95 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:04.377980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce2490-8e30-49de-84f5-d356792b8c95 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:04.455819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83fb6f-7eaa-41ac-b21b-b2878525a15e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:04.463335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 19/38] 10.222.0.71 () {60 vars in 1131 bytes} [Mon May 16 14:55:04 2022] POST /identity/v3/roles => generated 219 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:04.465231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09d72-f12b-46df-8d26-a8c9a42c0c61 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.465856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09d72-f12b-46df-8d26-a8c9a42c0c61 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.466259 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09d72-f12b-46df-8d26-a8c9a42c0c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.486747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-32e09d72-f12b-46df-8d26-a8c9a42c0c61 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:04.488181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-32e09d72-f12b-46df-8d26-a8c9a42c0c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:04.506130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:04.507107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.507511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.508906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.510388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:04.512202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:04.514097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 20/39] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:04.519826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:04.576082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 20/40] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:04.600089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e09d72-f12b-46df-8d26-a8c9a42c0c61 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:04.655323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce979d55-abc2-4627-9988-a598faaebeb4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:04.695033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864071de-9b1d-47b9-b3e0-0d92a00e4045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:04.697374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864071de-9b1d-47b9-b3e0-0d92a00e4045 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.697836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864071de-9b1d-47b9-b3e0-0d92a00e4045 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.698131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864071de-9b1d-47b9-b3e0-0d92a00e4045 None admin] PATH_INFO: `/v3/domains/default/users/6246a26f46e94e7389a45ee846fec1c9/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.701302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864071de-9b1d-47b9-b3e0-0d92a00e4045 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6246a26f46e94e7389a45ee846fec1c9, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:04.702769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864071de-9b1d-47b9-b3e0-0d92a00e4045 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:04.722628 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e09d72-f12b-46df-8d26-a8c9a42c0c61 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:04.755789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:04.756459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.756742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.757017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.757772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:04.759886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:04.781446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 21/41] 10.222.0.71 () {58 vars in 1361 bytes} [Mon May 16 14:55:04 2022] PUT /identity/v3/domains/default/users/6246a26f46e94e7389a45ee846fec1c9/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:04.782748 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73717d92-3249-46ba-8738-47e6ceaae372 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.783036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73717d92-3249-46ba-8738-47e6ceaae372 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.783313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73717d92-3249-46ba-8738-47e6ceaae372 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.784596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 22/42] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:04.790494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05495478-e3d7-4092-b5b7-5a1c6db4279b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:04.790771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05495478-e3d7-4092-b5b7-5a1c6db4279b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:04.791003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05495478-e3d7-4092-b5b7-5a1c6db4279b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:04.809220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-05495478-e3d7-4092-b5b7-5a1c6db4279b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:04.810809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-05495478-e3d7-4092-b5b7-5a1c6db4279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:04.829071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 21/43] 10.222.0.71 () {60 vars in 1131 bytes} [Mon May 16 14:55:04 2022] POST /identity/v3/roles => generated 223 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:04.901175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 23/44] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:04.925741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:05.037858 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f95de-1980-44b6-a4f6-2e72e5dd36d7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:05.071572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4b579-aed7-4316-b5ad-fc9c6fe59fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:05.072545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4b579-aed7-4316-b5ad-fc9c6fe59fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:05.072835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4b579-aed7-4316-b5ad-fc9c6fe59fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:05.073115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4b579-aed7-4316-b5ad-fc9c6fe59fc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:05.074329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4b579-aed7-4316-b5ad-fc9c6fe59fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:05.075865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4b579-aed7-4316-b5ad-fc9c6fe59fc3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:05.077677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 22/45] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:05.083407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05495478-e3d7-4092-b5b7-5a1c6db4279b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:05.178396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05495478-e3d7-4092-b5b7-5a1c6db4279b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:05.211903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2c7c3-268b-4dc5-b474-7064ac0267d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:05.212565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2c7c3-268b-4dc5-b474-7064ac0267d1 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:05.212854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2c7c3-268b-4dc5-b474-7064ac0267d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:05.213135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2c7c3-268b-4dc5-b474-7064ac0267d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:05.213852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2c7c3-268b-4dc5-b474-7064ac0267d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:05.215304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2c7c3-268b-4dc5-b474-7064ac0267d1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:05.280113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 24/46] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:55:05 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:07.036912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c2b23-331b-435a-91bd-c5d815aae002 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.037444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c2b23-331b-435a-91bd-c5d815aae002 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.037528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c2b23-331b-435a-91bd-c5d815aae002 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.038836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 23/47] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:07.044829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df797b6-c6bc-4b24-91d8-9030b93bab58 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.045277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df797b6-c6bc-4b24-91d8-9030b93bab58 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.045397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df797b6-c6bc-4b24-91d8-9030b93bab58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.062131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5df797b6-c6bc-4b24-91d8-9030b93bab58 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:07.063379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5df797b6-c6bc-4b24-91d8-9030b93bab58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:07.127493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 25/48] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:07.150592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c2b23-331b-435a-91bd-c5d815aae002 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:07.249326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c2b23-331b-435a-91bd-c5d815aae002 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:07.281004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:07.281801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.282091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.282373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.292796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:07.300699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:07.301651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:07.302571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 24/49] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:07.308639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df797b6-c6bc-4b24-91d8-9030b93bab58 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:07.413613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df797b6-c6bc-4b24-91d8-9030b93bab58 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:07.447177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209d2a9-0571-4622-86ec-638e95e77ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:07.447887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209d2a9-0571-4622-86ec-638e95e77ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.448179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209d2a9-0571-4622-86ec-638e95e77ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.448463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209d2a9-0571-4622-86ec-638e95e77ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.449223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209d2a9-0571-4622-86ec-638e95e77ab6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:07.449536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209d2a9-0571-4622-86ec-638e95e77ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:07.450929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209d2a9-0571-4622-86ec-638e95e77ab6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:07.458813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 26/50] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:07.464761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:07.495893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad146fa-49aa-4ae2-a1e4-c1dfc7a0f586 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.496186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad146fa-49aa-4ae2-a1e4-c1dfc7a0f586 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.496470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad146fa-49aa-4ae2-a1e4-c1dfc7a0f586 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.497914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 27/51] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:07.503020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867fcfe-93a9-4fcb-8b58-c84df8c98e6e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.503308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867fcfe-93a9-4fcb-8b58-c84df8c98e6e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.503587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867fcfe-93a9-4fcb-8b58-c84df8c98e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.519488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7867fcfe-93a9-4fcb-8b58-c84df8c98e6e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:07.520701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7867fcfe-93a9-4fcb-8b58-c84df8c98e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:07.562968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e8639-4fe8-4220-ba41-a320fd3576e6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:07.584485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 28/52] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:07.597191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:07.598059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.598343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.598620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] PATH_INFO: `/v3/users/6246a26f46e94e7389a45ee846fec1c9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.599894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] RBAC: Authorizing `identity:get_user(user_id=6246a26f46e94e7389a45ee846fec1c9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:07.601337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:07.603599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 25/53] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:07 2022] GET /identity/v3/users/6246a26f46e94e7389a45ee846fec1c9 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:07.608767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867fcfe-93a9-4fcb-8b58-c84df8c98e6e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:07.608970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:07.702036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19413-4efc-4ab4-9e21-3d5fbf2e0f8b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:07.702238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867fcfe-93a9-4fcb-8b58-c84df8c98e6e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:07.734791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:07.734949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:07.735569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.735687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.735882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.735968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.736159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.736259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.737324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:07.737424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:07.738753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:07.738838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:07.740379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 29/55] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:07.740470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 26/55] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:07.745910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:07.746025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:07.837847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3025aff-3e4b-4e06-94ce-e6db574bb35b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:07.839133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd1cfb-3731-4fa3-9888-09610b64ee3f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:07.871847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e2857-2ae8-4678-a5ee-c98757a9182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:07.872720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e2857-2ae8-4678-a5ee-c98757a9182f None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.873161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e2857-2ae8-4678-a5ee-c98757a9182f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.873441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e2857-2ae8-4678-a5ee-c98757a9182f None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.873538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ca449-100d-46d0-a98b-6111d5c139c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:07.874148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca449-100d-46d0-a98b-6111d5c139c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:07.874235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e2857-2ae8-4678-a5ee-c98757a9182f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:07.874436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca449-100d-46d0-a98b-6111d5c139c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:07.874720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca449-100d-46d0-a98b-6111d5c139c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:07.875564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ca449-100d-46d0-a98b-6111d5c139c9 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.domain.id=default, user.id=6246a26f46e94e7389a45ee846fec1c9) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:07.875673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e2857-2ae8-4678-a5ee-c98757a9182f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:07.875925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ca449-100d-46d0-a98b-6111d5c139c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:07.877359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ca449-100d-46d0-a98b-6111d5c139c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:07.883070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 27/56] 10.222.0.71 () {56 vars in 1322 bytes} [Mon May 16 14:55:07 2022] GET /identity/v3/role_assignments?user.id=6246a26f46e94e7389a45ee846fec1c9&scope.domain.id=default&role.id=1b3a0717a878438ea224dec0c7639471 => generated 530 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:07.972775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 30/57] 10.222.0.71 () {60 vars in 1140 bytes} [Mon May 16 14:55:07 2022] POST /identity/v3/projects => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:10.166744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9460-54e2-43ab-891c-fce6a57ddfc7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:10.167251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9460-54e2-43ab-891c-fce6a57ddfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:10.167251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9460-54e2-43ab-891c-fce6a57ddfc7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:10.168616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 28/58] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:10.173664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a1e9-e851-4575-839c-6b5c0cc49b29 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:10.174042 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a1e9-e851-4575-839c-6b5c0cc49b29 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:10.174224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a1e9-e851-4575-839c-6b5c0cc49b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:10.190821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-25a0a1e9-e851-4575-839c-6b5c0cc49b29 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:10.192052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-25a0a1e9-e851-4575-839c-6b5c0cc49b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:10.254681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 31/59] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:10.280708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9460-54e2-43ab-891c-fce6a57ddfc7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:10.379392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9460-54e2-43ab-891c-fce6a57ddfc7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:10.411339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e355e-01cc-4b00-bd5d-849d18d08592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:10.412144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e355e-01cc-4b00-bd5d-849d18d08592 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:10.412431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e355e-01cc-4b00-bd5d-849d18d08592 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:10.412707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e355e-01cc-4b00-bd5d-849d18d08592 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:10.413866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e355e-01cc-4b00-bd5d-849d18d08592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:10.416071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e355e-01cc-4b00-bd5d-849d18d08592 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:10.417730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 29/60] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:10.423124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a1e9-e851-4575-839c-6b5c0cc49b29 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:10.516173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a1e9-e851-4575-839c-6b5c0cc49b29 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:10.547768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21968c64-1bab-454c-8a1f-3ec15bfd56fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:10.548425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968c64-1bab-454c-8a1f-3ec15bfd56fa None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:10.548712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968c64-1bab-454c-8a1f-3ec15bfd56fa None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:10.548991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968c64-1bab-454c-8a1f-3ec15bfd56fa None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:10.549699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21968c64-1bab-454c-8a1f-3ec15bfd56fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:10.551157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21968c64-1bab-454c-8a1f-3ec15bfd56fa None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:10.656198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 32/61] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:55:10 2022] POST /identity/v3/users => generated 275 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:13.801526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3470-492a-4ffd-9b00-29ff6bda83d5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:13.802565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3470-492a-4ffd-9b00-29ff6bda83d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:13.803055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3470-492a-4ffd-9b00-29ff6bda83d5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:13.804672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 30/62] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:13.810887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496dce-3022-4de6-9470-d4e32d6c6b20 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:13.811307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496dce-3022-4de6-9470-d4e32d6c6b20 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:13.811482 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496dce-3022-4de6-9470-d4e32d6c6b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:13.836093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d7496dce-3022-4de6-9470-d4e32d6c6b20 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:13.838633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d7496dce-3022-4de6-9470-d4e32d6c6b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:13.942121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 33/63] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:13.978907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3470-492a-4ffd-9b00-29ff6bda83d5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.125455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3470-492a-4ffd-9b00-29ff6bda83d5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:14.175075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70964840-0ba6-4678-8b81-fe5ddfb909c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:14.176300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70964840-0ba6-4678-8b81-fe5ddfb909c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.176790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70964840-0ba6-4678-8b81-fe5ddfb909c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.177265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70964840-0ba6-4678-8b81-fe5ddfb909c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.188497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70964840-0ba6-4678-8b81-fe5ddfb909c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-70964840-0ba6-4678-8b81-fe5ddfb909c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:14.201268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:14.202225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:14.202997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:14.205362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 31/64] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:14.211588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7496dce-3022-4de6-9470-d4e32d6c6b20 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.218499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e05d-e5e6-4b57-a399-aa60d1044e86 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.219037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e05d-e5e6-4b57-a399-aa60d1044e86 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.219552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e05d-e5e6-4b57-a399-aa60d1044e86 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.221153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 32/65] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:14.226430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeccabf-7094-4c33-a487-b952eb68cc14 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.226972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeccabf-7094-4c33-a487-b952eb68cc14 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.227454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeccabf-7094-4c33-a487-b952eb68cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.244968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2aeccabf-7094-4c33-a487-b952eb68cc14 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:14.246502 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2aeccabf-7094-4c33-a487-b952eb68cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:14.320107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 33/66] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:14.327650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9dab4-c15b-433a-b283-c674620b729c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.328258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9dab4-c15b-433a-b283-c674620b729c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.328768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9dab4-c15b-433a-b283-c674620b729c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.330377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 34/67] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:14.336915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba11b62-4685-441f-a042-8a97b51b3e16 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.337619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba11b62-4685-441f-a042-8a97b51b3e16 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.338287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba11b62-4685-441f-a042-8a97b51b3e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.367069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3ba11b62-4685-441f-a042-8a97b51b3e16 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:14.369202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ba11b62-4685-441f-a042-8a97b51b3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:14.370224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7496dce-3022-4de6-9470-d4e32d6c6b20 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:14.408557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:14.409476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.409918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.410320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.411267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:14.411746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:14.413329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:14.423264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 34/68] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:14.424491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.455663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 35/69] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:14.455663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba11b62-4685-441f-a042-8a97b51b3e16 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.550107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba11b62-4685-441f-a042-8a97b51b3e16 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:14.559067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5a415-cb24-4fa4-8f10-9feceefb4259 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:14.584388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:14.585426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.586000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.586604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.597196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022894df-cccc-4537-8a2a-96c769b01f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:14.598038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022894df-cccc-4537-8a2a-96c769b01f4f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.598333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022894df-cccc-4537-8a2a-96c769b01f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.598611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022894df-cccc-4537-8a2a-96c769b01f4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.601777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:14.603375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:14.607730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 36/70] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:14 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:14.607730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.609327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022894df-cccc-4537-8a2a-96c769b01f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-022894df-cccc-4537-8a2a-96c769b01f4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:14.617608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:14.619236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:14.620065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 35/71] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:14.622095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d935-1f7d-464e-8d23-9f7d91572be8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.622095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d935-1f7d-464e-8d23-9f7d91572be8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.622095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d935-1f7d-464e-8d23-9f7d91572be8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.623093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 36/72] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:14.624203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d935-1f7d-464e-8d23-9f7d91572be8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.718272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137959b-107c-4704-b12b-b87f35ec5de1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:14.724915 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d935-1f7d-464e-8d23-9f7d91572be8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:14.754742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:14.755555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.755939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.756387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.759834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:14.760490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.760782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.761067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.766569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:14.774939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:14.776137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 37/73] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:14.781566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.835185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:14.836946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:14.877408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc145-e480-47dd-9e9b-d19e0a38b608 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:14.909063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5df1fa-8dae-4a7f-8a47-272d3dd424df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:14.909734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df1fa-8dae-4a7f-8a47-272d3dd424df None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.910027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df1fa-8dae-4a7f-8a47-272d3dd424df None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.910314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df1fa-8dae-4a7f-8a47-272d3dd424df None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.911062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5df1fa-8dae-4a7f-8a47-272d3dd424df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:14.911375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5df1fa-8dae-4a7f-8a47-272d3dd424df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:14.912797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5df1fa-8dae-4a7f-8a47-272d3dd424df None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:14.925109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 38/74] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:14.926348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3d6f0-8b12-4add-b9a5-544c9a8047d0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:14.926644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3d6f0-8b12-4add-b9a5-544c9a8047d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:14.926935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3d6f0-8b12-4add-b9a5-544c9a8047d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:14.935684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 37/75] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:14.935684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:14.943530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fbc3d6f0-8b12-4add-b9a5-544c9a8047d0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:14.944758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fbc3d6f0-8b12-4add-b9a5-544c9a8047d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:15.008682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 39/76] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:15.009871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964ccc4-d8d6-4522-a1d9-be6c37bc2b6f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.010113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964ccc4-d8d6-4522-a1d9-be6c37bc2b6f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.010403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964ccc4-d8d6-4522-a1d9-be6c37bc2b6f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.011826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 40/77] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:15.016481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964ccc4-d8d6-4522-a1d9-be6c37bc2b6f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.028415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91c74-7ab7-438d-b4ad-1204012ac4e7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.060900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.061565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.061841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.062111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.062854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:15.063153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.064636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.072196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 38/78] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:15.072998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.115260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964ccc4-d8d6-4522-a1d9-be6c37bc2b6f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.148747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd028e-da0a-4001-802e-1f524368381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.149651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd028e-da0a-4001-802e-1f524368381a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.149948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd028e-da0a-4001-802e-1f524368381a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.150211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd028e-da0a-4001-802e-1f524368381a None admin] PATH_INFO: `/v3/users/6246a26f46e94e7389a45ee846fec1c9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.151387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd028e-da0a-4001-802e-1f524368381a None admin] RBAC: Authorizing `identity:get_user(user_id=6246a26f46e94e7389a45ee846fec1c9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.152875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd028e-da0a-4001-802e-1f524368381a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.154498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 41/79] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/users/6246a26f46e94e7389a45ee846fec1c9 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:15.156874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2395c-f1e0-4a49-8fee-c4a4294a23a9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.157166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2395c-f1e0-4a49-8fee-c4a4294a23a9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.157447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2395c-f1e0-4a49-8fee-c4a4294a23a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.169286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98a417-db11-42b5-b7ef-30bd87adf0ab None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.173018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ebc2395c-f1e0-4a49-8fee-c4a4294a23a9 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:15.174190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ebc2395c-f1e0-4a49-8fee-c4a4294a23a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:15.202190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.203042 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.203331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.203602 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.204807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.206247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.207964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 39/80] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:15.208724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.238204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 42/81] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:15.239038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2395c-f1e0-4a49-8fee-c4a4294a23a9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.316654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642987-7509-4c3c-86fc-c251384796cb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.329946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2395c-f1e0-4a49-8fee-c4a4294a23a9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.350003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.350792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.351072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.351343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.361425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:15.363497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.364381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.364664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.364938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.366097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.367530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:15.369107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:15.370116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:15.371091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:15.371838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 43/82] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:15.371838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.374998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 40/83] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:15.375759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.465576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109d03b-cf20-4c65-bf40-615d8de50489 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.469975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366265e-2c66-47b8-bb98-e0960199a2c8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.497021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.497669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.497958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.498265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.501473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.506114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.506114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.506114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.506114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=160bfc163c5d49f9a9bae22a11e7f2e7) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:15.506114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.506114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.513986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 41/84] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/role_assignments?user.id=160bfc163c5d49f9a9bae22a11e7f2e7&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:15.514775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.569093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.570516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.621678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cdbd-2814-4033-959f-1414fbd885f2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.657593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.658398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.658682 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.658957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.660183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.661680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.668234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 42/85] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:15.668234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.686875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 44/86] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:15.686875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.774316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d6151-9e08-414b-a2d1-cedaa4099983 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.786289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b35c8-23a4-43b1-b9b8-25dda65f55b9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.809221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.809889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.810180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.810461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.824365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.824973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.825263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.825545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.826300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:15.826609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.828030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.836463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 45/87] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:15.837864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:15.886150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.887656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.937647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f80e-ad5f-4229-92e8-49b33ba5f5b9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:15.969946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:15.970621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:15.970903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:15.971178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:15.971900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:15.973338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:15.984610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 43/88] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:15.989978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:16.082519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509ebb3-cd8b-4bb0-94cc-b8e2d94e3690 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:16.099658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 46/89] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:55:15 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:16.099658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:16.131688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:16.133297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:16.133750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:16.134274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:16.136496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:16.139187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:16.142380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 44/90] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:15 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:16.144074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:16.300651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9176c-6114-4594-91a2-dd131e996d90 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:16.325341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabc04f-e097-4322-ba81-56bfea128b55 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:16.356221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:16.357028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:16.357308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:16.357578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:16.359430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:16.360905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:16.362537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 47/91] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:16 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:16.363808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:16.374540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:16.375405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:16.375716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:16.375992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:16.377181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:16.378615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:16.380355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 45/92] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:16 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:16.385553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:16.498906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edb12-038c-47b7-9caf-85121d08b1e9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:16.506173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b810076-2ca1-423f-aaec-883178bbd2cf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:16.541720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:16.542377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:16.542659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:16.542931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:16.543206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:16.544003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:16.544293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:16.544575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:16.545421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=6246a26f46e94e7389a45ee846fec1c9) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:16.545915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:16.548270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:16.555565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 48/93] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:16 2022] GET /identity/v3/role_assignments?user.id=6246a26f46e94e7389a45ee846fec1c9&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:16.557056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:16.661462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:16.662889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:16.775484 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 46/94] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:16.781419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:16.783973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eeb0a-552c-489b-8f08-654cc4daecc2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:16.851664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:16.851664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:16.851664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:16.851664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:16.852025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] RBAC: Adding query filter params (role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=160bfc163c5d49f9a9bae22a11e7f2e7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:16.852025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:16.852025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:16.855265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 49/95] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:16 2022] GET /identity/v3/role_assignments?user.id=160bfc163c5d49f9a9bae22a11e7f2e7&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:16.905605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e2280-0dd4-476a-a95f-66001b692c8f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:16.952678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88d5ce-7ce8-49df-b5c1-e6b74764449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:16.953798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88d5ce-7ce8-49df-b5c1-e6b74764449d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:16.954230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88d5ce-7ce8-49df-b5c1-e6b74764449d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:16.954649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88d5ce-7ce8-49df-b5c1-e6b74764449d None admin] PATH_INFO: `/v3/projects/f510cfe1ac764712bf859c01884e55f7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:16.956021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88d5ce-7ce8-49df-b5c1-e6b74764449d None admin] RBAC: Authorizing `identity:get_project(project_id=f510cfe1ac764712bf859c01884e55f7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:16.957622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88d5ce-7ce8-49df-b5c1-e6b74764449d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:16.959502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 47/96] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:16 2022] GET /identity/v3/projects/f510cfe1ac764712bf859c01884e55f7 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:16.966138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:17.087744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78277bc-db47-4ddb-acae-dac7ef7157ff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:17.126258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ccf73-1930-498e-8de0-b1398de83370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:17.126923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ccf73-1930-498e-8de0-b1398de83370 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:17.127209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ccf73-1930-498e-8de0-b1398de83370 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:17.127488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ccf73-1930-498e-8de0-b1398de83370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:17.128354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ccf73-1930-498e-8de0-b1398de83370 None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=f510cfe1ac764712bf859c01884e55f7, user.id=160bfc163c5d49f9a9bae22a11e7f2e7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:17.128669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ccf73-1930-498e-8de0-b1398de83370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:17.130125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ccf73-1930-498e-8de0-b1398de83370 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:17.137013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 50/97] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:16 2022] GET /identity/v3/role_assignments?user.id=160bfc163c5d49f9a9bae22a11e7f2e7&scope.project.id=f510cfe1ac764712bf859c01884e55f7&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:18.086411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1eff6-433c-4d40-8284-08c29b9ccb3e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:18.087500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1eff6-433c-4d40-8284-08c29b9ccb3e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:18.088030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1eff6-433c-4d40-8284-08c29b9ccb3e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:18.089922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 48/98] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:18.095828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c1f6-1b27-43de-87d8-1026e62ae810 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:18.096667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c1f6-1b27-43de-87d8-1026e62ae810 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:18.097154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c1f6-1b27-43de-87d8-1026e62ae810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:18.120704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1a00c1f6-1b27-43de-87d8-1026e62ae810 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:18.122260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a00c1f6-1b27-43de-87d8-1026e62ae810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:18.209158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 51/99] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:18.243795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1eff6-433c-4d40-8284-08c29b9ccb3e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:18.365347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1eff6-433c-4d40-8284-08c29b9ccb3e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:18.405330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:18.406148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:18.406434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:18.406709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:18.420853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:18.428605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:18.429509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:18.430729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 49/100] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:18.436210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00c1f6-1b27-43de-87d8-1026e62ae810 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:18.610097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00c1f6-1b27-43de-87d8-1026e62ae810 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:18.656429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:18.657039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:18.657324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:18.657731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:18.658356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:18.658661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:18.660081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:18.668960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 52/101] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:18.674506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:18.835160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1b12-41de-42cf-be43-94bc08ddd261 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:18.886807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:18.888102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:18.888592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:18.889069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:18.890500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:18.892165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:18.894350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 50/102] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:18 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:18.901510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:19.037283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0702e0-e64c-480a-8c3e-89326540aef0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:19.075930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3436a-801a-46b7-88bd-08f9e1185d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:19.076801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3436a-801a-46b7-88bd-08f9e1185d64 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:19.077090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3436a-801a-46b7-88bd-08f9e1185d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:19.077368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3436a-801a-46b7-88bd-08f9e1185d64 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:19.078577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3436a-801a-46b7-88bd-08f9e1185d64 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:19.080053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3436a-801a-46b7-88bd-08f9e1185d64 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:19.081967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 53/103] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:18 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:19.092678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:19.240092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328c43c-54af-4aae-9556-ccde5ffd2126 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:19.283786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:19.285129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:19.285616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:19.286082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/160bfc163c5d49f9a9bae22a11e7f2e7/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:19.288328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] RBAC: Authorizing `identity:create_grant(project_id=712891263b4f468fb3ce3dda3aeaed04, user_id=160bfc163c5d49f9a9bae22a11e7f2e7, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:19.289978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:19.366610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b62d04-e8ba-49d1-ab15-b07a82f8cd48 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:19.367130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b62d04-e8ba-49d1-ab15-b07a82f8cd48 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:19.367211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b62d04-e8ba-49d1-ab15-b07a82f8cd48 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:19.369466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 54/104] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:19.374686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe32fd-c3e4-41cb-b0cf-a27074fc1b5e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:19.374986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe32fd-c3e4-41cb-b0cf-a27074fc1b5e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:19.375276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe32fd-c3e4-41cb-b0cf-a27074fc1b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:19.384136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 51/105] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:19 2022] PUT /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/160bfc163c5d49f9a9bae22a11e7f2e7/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:19.401106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-affe32fd-c3e4-41cb-b0cf-a27074fc1b5e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:19.402412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-affe32fd-c3e4-41cb-b0cf-a27074fc1b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:19.494248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 55/106] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:19.528731 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:19.644912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1348550-f74c-4e36-a526-9fbe7b04cc62 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:19.688522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:19.689700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:19.690189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:19.690667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:19.705916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:19.714953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:19.715914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:19.716764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:19.719265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 52/107] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:19.725008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe32fd-c3e4-41cb-b0cf-a27074fc1b5e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:19.851442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe32fd-c3e4-41cb-b0cf-a27074fc1b5e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:19.898842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:19.899954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:19.900446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:19.900921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:19.901924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:19.902435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:19.904033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:19.915215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 56/108] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:19.921582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:20.041113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8de878-8564-4201-9fdd-f18222129c8e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:20.082746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da321b36-f637-4e59-876a-50405f9b27fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:20.084033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da321b36-f637-4e59-876a-50405f9b27fc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.084526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da321b36-f637-4e59-876a-50405f9b27fc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.084996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da321b36-f637-4e59-876a-50405f9b27fc None admin] PATH_INFO: `/v3/users/12b32173f72043d684a71c7411693336` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.107715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da321b36-f637-4e59-876a-50405f9b27fc None admin] RBAC: Authorizing `identity:get_user(user_id=12b32173f72043d684a71c7411693336)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:20.109993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da321b36-f637-4e59-876a-50405f9b27fc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:20.112061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 53/109] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:19 2022] GET /identity/v3/users/12b32173f72043d684a71c7411693336 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:20.117876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:20.237681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59866f-59aa-489e-a4e5-af804fddf4a9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:20.287529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcd592-0545-4f30-b768-185546391313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:20.288642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcd592-0545-4f30-b768-185546391313 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.289132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcd592-0545-4f30-b768-185546391313 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.289606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcd592-0545-4f30-b768-185546391313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.347993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367fc3c-0695-40b5-8f48-146636bd1ba6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.348662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367fc3c-0695-40b5-8f48-146636bd1ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.349151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367fc3c-0695-40b5-8f48-146636bd1ba6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.350714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 54/110] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:20.357280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df2d4-e840-465a-aec6-d4cbe0fefc08 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.357829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df2d4-e840-465a-aec6-d4cbe0fefc08 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.358458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582df2d4-e840-465a-aec6-d4cbe0fefc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.385193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-582df2d4-e840-465a-aec6-d4cbe0fefc08 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:20.388272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-582df2d4-e840-465a-aec6-d4cbe0fefc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:20.392633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcd592-0545-4f30-b768-185546391313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:20.394704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcd592-0545-4f30-b768-185546391313 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:20.504479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 55/111] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:20.506243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990f405-5c2a-4e23-965f-9a34b2a44a41 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.506746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990f405-5c2a-4e23-965f-9a34b2a44a41 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.507226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990f405-5c2a-4e23-965f-9a34b2a44a41 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.508995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 56/112] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:20.511237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8448a-fbf7-4fb4-ab33-a79c0bd0eb15 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.511842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8448a-fbf7-4fb4-ab33-a79c0bd0eb15 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.512346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8448a-fbf7-4fb4-ab33-a79c0bd0eb15 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.513995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 57/113] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:20.519801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54add9-fddf-485c-b282-0af7e5c5acc7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.520375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54add9-fddf-485c-b282-0af7e5c5acc7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.520872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54add9-fddf-485c-b282-0af7e5c5acc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.522158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 57/114] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:20.523514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729c20d-f350-4231-b953-2624fad0f815 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.523813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729c20d-f350-4231-b953-2624fad0f815 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.524094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729c20d-f350-4231-b953-2624fad0f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.544649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ee54add9-fddf-485c-b282-0af7e5c5acc7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:20.545022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4729c20d-f350-4231-b953-2624fad0f815 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:20.545829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ee54add9-fddf-485c-b282-0af7e5c5acc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:20.546611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4729c20d-f350-4231-b953-2624fad0f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:20.616681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 58/115] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:20.623073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729c20d-f350-4231-b953-2624fad0f815 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:20.624211 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 58/116] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:20.624897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54add9-fddf-485c-b282-0af7e5c5acc7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:20.749164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54add9-fddf-485c-b282-0af7e5c5acc7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:20.750199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729c20d-f350-4231-b953-2624fad0f815 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:20.783139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:20.784043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.784331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.784610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.785406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:20.785774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:20.786192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.786474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.786750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.787291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:20.787929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:20.789138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 59/117] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:20 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:20.789739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:20.794622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:20.800799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 59/118] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:20.800799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:20.896798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047166c-51ad-4b23-9bee-7b39f5949a88 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:20.900224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c59267-e51d-43fb-a6ba-a2134a0a787a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:20.933303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001d67b-6e54-4daa-9c77-2257564a6b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:20.934148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d67b-6e54-4daa-9c77-2257564a6b0f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.934437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d67b-6e54-4daa-9c77-2257564a6b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.934718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d67b-6e54-4daa-9c77-2257564a6b0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.936908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:20.937919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.938418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.938888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.944618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001d67b-6e54-4daa-9c77-2257564a6b0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:20.949334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5001d67b-6e54-4daa-9c77-2257564a6b0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:20.952954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:20.953811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:20.954572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 60/119] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:20.955720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819d388-b71b-4b48-8f26-32b94272b546 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:20.955720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819d388-b71b-4b48-8f26-32b94272b546 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:20.955979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819d388-b71b-4b48-8f26-32b94272b546 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:20.957243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 61/120] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:20.957942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819d388-b71b-4b48-8f26-32b94272b546 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:20.961420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:20.962272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:20.963601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:20.964727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 60/121] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:20.965521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.074408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819d388-b71b-4b48-8f26-32b94272b546 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.075949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740e63f-eadb-40ad-92b0-2df4420c0463 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.108768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.109419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.109701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.109977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.110807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] RBAC: Adding query filter params (role.id=1a29288217364c61b38b69e4e823a4d5, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=12b32173f72043d684a71c7411693336) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:21.111116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.111214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf88f55-053f-4090-937d-1ca54d82cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.111898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf88f55-053f-4090-937d-1ca54d82cc0d None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.112184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf88f55-053f-4090-937d-1ca54d82cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.112458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf88f55-053f-4090-937d-1ca54d82cc0d None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.112604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.113161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf88f55-053f-4090-937d-1ca54d82cc0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.114783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf88f55-053f-4090-937d-1ca54d82cc0d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.123962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 62/122] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:20 2022] GET /identity/v3/role_assignments?user.id=12b32173f72043d684a71c7411693336&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=1a29288217364c61b38b69e4e823a4d5 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.123962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.239663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 61/123] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:55:20 2022] POST /identity/v3/projects => generated 302 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:21.248122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2596ed-de60-46a8-a99b-84b7d02d98ad None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.250842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2596ed-de60-46a8-a99b-84b7d02d98ad None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.250842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2596ed-de60-46a8-a99b-84b7d02d98ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.267659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f731-44d2-43e5-93e2-36e0765e305b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.290103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ac2596ed-de60-46a8-a99b-84b7d02d98ad None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:21.295660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ac2596ed-de60-46a8-a99b-84b7d02d98ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:21.302987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.307728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.307728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.307728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.307728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:21.307728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.307728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.317578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 63/124] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.318590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.399379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 62/125] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:21.399537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2596ed-de60-46a8-a99b-84b7d02d98ad None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.445009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b221311-a074-4538-9a29-5b96d33003dc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.487737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.487737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.487737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.487737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.488109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:21.488109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.488242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.498394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 64/126] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.499153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.510496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2596ed-de60-46a8-a99b-84b7d02d98ad None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.551360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.551360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.551360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.551360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.551810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.552853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.555668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 63/127] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.555668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.607658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2122e-a498-4844-bdaa-54eaf2f35a33 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.644971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.645787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.646074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.646357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.658385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:21.666616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:21.667517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:21.668336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 65/128] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:21.669438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.672371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e631163-a029-4004-bab3-c80068ba6cfd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.713327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.713327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.713327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.713327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.714865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.714865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.716249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 64/129] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.717087 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.788662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d562-fec8-4a75-a432-ed2d42e5499d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.827654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34458266-5fb9-4a95-8eff-474de480a145 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.827838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.827838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.827838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.827838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.831652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.831938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.835477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 66/130] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.835477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.867443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.868194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.868480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.868782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.869637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:21.869956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.871570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.883666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 65/131] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.883666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:21.951208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226301ff-1ba8-49f8-8358-c4db337496b6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:21.990079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:21.991168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:21.991457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:21.991763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] PATH_INFO: `/v3/projects/f510cfe1ac764712bf859c01884e55f7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:21.992985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] RBAC: Authorizing `identity:get_project(project_id=f510cfe1ac764712bf859c01884e55f7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:21.995214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:21.997706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 67/132] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/projects/f510cfe1ac764712bf859c01884e55f7 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:21.998513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:22.007335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11296104-7734-4f3e-9daa-4634c30cbd1f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:22.046108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:22.047056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:22.047340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:22.047613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/160bfc163c5d49f9a9bae22a11e7f2e7/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:22.049949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=712891263b4f468fb3ce3dda3aeaed04, user_id=160bfc163c5d49f9a9bae22a11e7f2e7, role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:22.051673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:22.114121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9584bc-d936-43b7-8b42-a8b50e5beceb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:22.155285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:22.156204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:22.156488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:22.156765 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] PATH_INFO: `/v3/users/6246a26f46e94e7389a45ee846fec1c9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:22.157959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=6246a26f46e94e7389a45ee846fec1c9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:22.159483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:22.167189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 66/133] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:21 2022] PUT /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/160bfc163c5d49f9a9bae22a11e7f2e7/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:22.168037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:22.171999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 68/134] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:21 2022] GET /identity/v3/users/6246a26f46e94e7389a45ee846fec1c9 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:22.178639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:22.361977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a873-1d2b-4b03-a0a8-1417742d4fd9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:22.405806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:22.406692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:22.406975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:22.407439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:22.410040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:22.410650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:22.413216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 69/135] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:22 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:22.421965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:22.480304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d50bda-db24-41a5-be92-4a05811afff3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:22.557262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0114ad-a247-44db-9019-6d7ad5b15add None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:22.596984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:22.598279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:22.598744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:22.599198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] PATH_INFO: `/v3/projects/f510cfe1ac764712bf859c01884e55f7/users/160bfc163c5d49f9a9bae22a11e7f2e7/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:22.600189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d673ea-972b-4cba-9786-cae467382acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:22.603863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d673ea-972b-4cba-9786-cae467382acb None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:22.603863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d673ea-972b-4cba-9786-cae467382acb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:22.603863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d673ea-972b-4cba-9786-cae467382acb None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/6246a26f46e94e7389a45ee846fec1c9/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:22.604441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d673ea-972b-4cba-9786-cae467382acb None admin] RBAC: Authorizing `identity:create_grant(project_id=712891263b4f468fb3ce3dda3aeaed04, user_id=6246a26f46e94e7389a45ee846fec1c9, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:22.605523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] RBAC: Authorizing `identity:create_grant(project_id=f510cfe1ac764712bf859c01884e55f7, user_id=160bfc163c5d49f9a9bae22a11e7f2e7, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:22.607109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d673ea-972b-4cba-9786-cae467382acb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:22.607364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:22.681979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 70/136] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:22 2022] PUT /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/6246a26f46e94e7389a45ee846fec1c9/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:22.683731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7258d57-b571-4ede-9530-d2914f2cbe4c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:22.683919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7258d57-b571-4ede-9530-d2914f2cbe4c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:22.684193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7258d57-b571-4ede-9530-d2914f2cbe4c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:22.685668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 71/137] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:22.695379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f75c0-61f3-45aa-813c-43aa4e274dbb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:22.695728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f75c0-61f3-45aa-813c-43aa4e274dbb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:22.696012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f75c0-61f3-45aa-813c-43aa4e274dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:22.711811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 67/138] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:22 2022] PUT /identity/v3/projects/f510cfe1ac764712bf859c01884e55f7/users/160bfc163c5d49f9a9bae22a11e7f2e7/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:22.728050 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-084f75c0-61f3-45aa-813c-43aa4e274dbb None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:22.729893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-084f75c0-61f3-45aa-813c-43aa4e274dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:22.826714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 72/139] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 16 14:55:22.866244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:23.005506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bc135-0137-4bfb-be28-b03de036976b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:23.043128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:23.043964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:23.044297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:23.044577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:23.060537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:23.068392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:23.069254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:23.070030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 68/140] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:23.076245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f75c0-61f3-45aa-813c-43aa4e274dbb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:23.186776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f75c0-61f3-45aa-813c-43aa4e274dbb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:23.225504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:23.226555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:23.227054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:23.227530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:23.228555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:23.229166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:23.230753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:23.246684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 73/141] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:23.251922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:23.386988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f89dc5-6932-43e6-b724-0cdf9c398326 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:23.440341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:23.441226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:23.441506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:23.441782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:23.442997 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:23.444461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:23.446250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 69/142] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:23 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:23.451772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:23.611681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea342a90-e172-4b1b-b303-58140ab7ae55 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:23.661419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:23.662414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:23.662916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:23.663393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:23.753726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:23.755537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:23.888515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 74/143] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:23.893895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:24.015719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534f7c-340d-4682-a44f-3c0afb03da92 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:24.060642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39de3cc-ebc0-40b7-aea3-04b9c4c276d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:24.061496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de3cc-ebc0-40b7-aea3-04b9c4c276d2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:24.061778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de3cc-ebc0-40b7-aea3-04b9c4c276d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:24.062048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de3cc-ebc0-40b7-aea3-04b9c4c276d2 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:24.064280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39de3cc-ebc0-40b7-aea3-04b9c4c276d2 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:24.065827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39de3cc-ebc0-40b7-aea3-04b9c4c276d2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:24.067708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 70/144] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:23 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:24.073823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:24.171841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1c6a-e579-42f2-8ac8-c94cfe577a06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:24.215891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54dd2e-5f4f-4f62-88c5-0284f9a6fbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:24.216616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54dd2e-5f4f-4f62-88c5-0284f9a6fbbd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:24.216900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54dd2e-5f4f-4f62-88c5-0284f9a6fbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:24.217272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54dd2e-5f4f-4f62-88c5-0284f9a6fbbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:24.218117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54dd2e-5f4f-4f62-88c5-0284f9a6fbbd None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=160bfc163c5d49f9a9bae22a11e7f2e7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:24.218426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54dd2e-5f4f-4f62-88c5-0284f9a6fbbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:24.219899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54dd2e-5f4f-4f62-88c5-0284f9a6fbbd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:24.226341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 75/145] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:24 2022] GET /identity/v3/role_assignments?user.id=160bfc163c5d49f9a9bae22a11e7f2e7&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:24.483928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7e84-bba7-42c0-8d68-46066dfec0d4 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:24.484430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7e84-bba7-42c0-8d68-46066dfec0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:24.484503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7e84-bba7-42c0-8d68-46066dfec0d4 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:24.486150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 71/146] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:24.491345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526b270-717d-46eb-a73e-5a5a4a226afb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:24.491653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526b270-717d-46eb-a73e-5a5a4a226afb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:24.491935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526b270-717d-46eb-a73e-5a5a4a226afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:24.512718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f526b270-717d-46eb-a73e-5a5a4a226afb None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:24.513940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f526b270-717d-46eb-a73e-5a5a4a226afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:24.606823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 76/147] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:24.632633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd7e84-bba7-42c0-8d68-46066dfec0d4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:24.749443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd7e84-bba7-42c0-8d68-46066dfec0d4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:24.788410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:24.789214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:24.789500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:24.789783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:24.804512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:24.813448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:24.815509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:24.817463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 72/148] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:24.824929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526b270-717d-46eb-a73e-5a5a4a226afb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:24.975607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526b270-717d-46eb-a73e-5a5a4a226afb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.033777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de11b5c-2e3c-4a7d-bc35-1249d3a391cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:25.034836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de11b5c-2e3c-4a7d-bc35-1249d3a391cd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.035342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de11b5c-2e3c-4a7d-bc35-1249d3a391cd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.035865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de11b5c-2e3c-4a7d-bc35-1249d3a391cd None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.036872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de11b5c-2e3c-4a7d-bc35-1249d3a391cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:25.037439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de11b5c-2e3c-4a7d-bc35-1249d3a391cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:25.039103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de11b5c-2e3c-4a7d-bc35-1249d3a391cd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:25.055125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 77/149] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:25.061037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:25.231138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672633e5-ab56-4a3f-a96c-2917535b41ac None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.274958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:25.276528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.276810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.277075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] PATH_INFO: `/v3/users/12b32173f72043d684a71c7411693336` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.284494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] RBAC: Authorizing `identity:get_user(user_id=12b32173f72043d684a71c7411693336)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:25.285902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:25.287597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 73/150] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:25 2022] GET /identity/v3/users/12b32173f72043d684a71c7411693336 => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:25.287809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5b2fa-763a-4f55-b432-a35b555b9593 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.288291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5b2fa-763a-4f55-b432-a35b555b9593 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.288787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5b2fa-763a-4f55-b432-a35b555b9593 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.290588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 78/151] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:25.295518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:25.295955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca78919-2e83-464c-a5b7-4670b69c93d8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.296472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca78919-2e83-464c-a5b7-4670b69c93d8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.296985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca78919-2e83-464c-a5b7-4670b69c93d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.324468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6ca78919-2e83-464c-a5b7-4670b69c93d8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:25.325636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6ca78919-2e83-464c-a5b7-4670b69c93d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:25.434729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df535488-5cb1-475d-b93d-007560ad0676 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.459129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 79/152] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:25.460720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27de41-c9ab-4930-844c-c39bb6631719 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.461142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27de41-c9ab-4930-844c-c39bb6631719 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.461816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27de41-c9ab-4930-844c-c39bb6631719 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.464137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 80/153] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:25.473516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396df22-2c3d-4e74-bbe1-de09213f54de None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.474025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396df22-2c3d-4e74-bbe1-de09213f54de None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.474438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396df22-2c3d-4e74-bbe1-de09213f54de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.492019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f396df22-2c3d-4e74-bbe1-de09213f54de None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:25.493387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f396df22-2c3d-4e74-bbe1-de09213f54de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:25.497741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:25.498613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.498899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.499179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.501124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:25.502694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:25.504808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 74/154] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:25 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:25.511989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:25.566470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 81/155] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:25.567502 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396df22-2c3d-4e74-bbe1-de09213f54de None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:25.620550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58248df7-cc9a-4354-9a83-bb7027e807fd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.657074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:25.658138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.658569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.658988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.660369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:25.662028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:25.664196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 75/156] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:25.669710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:25.669852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396df22-2c3d-4e74-bbe1-de09213f54de None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.705711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:25.706597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.706875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.707146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/12b32173f72043d684a71c7411693336/roles/1a29288217364c61b38b69e4e823a4d5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.709139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=712891263b4f468fb3ce3dda3aeaed04, user_id=12b32173f72043d684a71c7411693336, role_id=1a29288217364c61b38b69e4e823a4d5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:25.710603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:25.776237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c31e7e-2fcd-4be5-bc68-3ed3c57b2e4c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.782768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 82/157] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:25 2022] PUT /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04/users/12b32173f72043d684a71c7411693336/roles/1a29288217364c61b38b69e4e823a4d5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:25.783573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:25.819826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fc498-0b3d-472d-8f30-cd9d59cc6419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:25.820713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fc498-0b3d-472d-8f30-cd9d59cc6419 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.821066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fc498-0b3d-472d-8f30-cd9d59cc6419 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.821406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fc498-0b3d-472d-8f30-cd9d59cc6419 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.840302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fc498-0b3d-472d-8f30-cd9d59cc6419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-180fc498-0b3d-472d-8f30-cd9d59cc6419 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:25.850787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:25.852068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 76/158] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:25.857340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4350-9c32-473d-95c8-3d51a0c116ad None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.858272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4350-9c32-473d-95c8-3d51a0c116ad None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.858272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4350-9c32-473d-95c8-3d51a0c116ad None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.858272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 77/159] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:25.862520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a4350-9c32-473d-95c8-3d51a0c116ad None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:25.948831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e0364-86b6-4a57-b4ab-868aa104a6e2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.988934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a4350-9c32-473d-95c8-3d51a0c116ad None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:25.994762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:25.996014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:25.996522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:25.996858 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:25.997680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:25.999204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:26.030383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e6f46-c20c-4402-bc37-25af332d5290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:26.031056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e6f46-c20c-4402-bc37-25af332d5290 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.031338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e6f46-c20c-4402-bc37-25af332d5290 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.031609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e6f46-c20c-4402-bc37-25af332d5290 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.032657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e6f46-c20c-4402-bc37-25af332d5290 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:26.032963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e6f46-c20c-4402-bc37-25af332d5290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:26.034398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e6f46-c20c-4402-bc37-25af332d5290 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:26.043772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 78/160] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:26.047139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dd44a-033b-46d8-9f61-3ab80b4f34c2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.047139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dd44a-033b-46d8-9f61-3ab80b4f34c2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.047139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dd44a-033b-46d8-9f61-3ab80b4f34c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.062259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a70dd44a-033b-46d8-9f61-3ab80b4f34c2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:26.063442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a70dd44a-033b-46d8-9f61-3ab80b4f34c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:26.100004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 83/161] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:55:25 2022] POST /identity/v3/users => generated 283 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:26.101410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.185971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 79/162] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:26.199012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed0d62-8e61-4aa0-8c40-1e058af63f7a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.199322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed0d62-8e61-4aa0-8c40-1e058af63f7a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.199562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed0d62-8e61-4aa0-8c40-1e058af63f7a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.201197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 80/163] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:26.221229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b09e-9499-4faa-a0dc-8a13a5ea9f18 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.221537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b09e-9499-4faa-a0dc-8a13a5ea9f18 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.221789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b09e-9499-4faa-a0dc-8a13a5ea9f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.244559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8e67b09e-9499-4faa-a0dc-8a13a5ea9f18 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:26.245782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8e67b09e-9499-4faa-a0dc-8a13a5ea9f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:26.264056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5b793-38f0-4114-b141-d21e719cc8ca None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:26.309753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:26.313940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.313940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.313940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.313940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:26.314839 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:26.317040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 84/164] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:26.318185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.339656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 81/165] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:26.339656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b09e-9499-4faa-a0dc-8a13a5ea9f18 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.460233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b09e-9499-4faa-a0dc-8a13a5ea9f18 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:26.472742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0ae0d-6136-4a5c-a107-9ac0831af29d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:26.500105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:26.501316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.501834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.502245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.513000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:26.514155 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.514275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.514643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.526085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:26.533860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:26.535004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:26.535822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:26.536700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 85/166] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:26.537185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.577000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:26.578826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:26.637265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e41bb-01df-4b93-bd67-7caa2006cea3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:26.671645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:26.672550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.672887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.673252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.682022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 82/167] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:26.683405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.684001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:26.691963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:26.692894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 86/168] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:26.699328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.798786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1afdb6-e815-4cfd-a038-4726d00b4ae3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:26.801935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424abe6-673f-4e95-a887-973e34a3ea05 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:26.840782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:26.841487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.841795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.842067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.842827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:26.843124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:26.844555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:26.848505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:26.849374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:26.849647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:26.850500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:26.851884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:26.852876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:26.855223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 87/169] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:26.856019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.857142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 83/170] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:26.861957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:26.977788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6f0d-24f4-4760-976f-dfdd5dcb7a65 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:26.997045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ff2b-0760-4881-9ba3-a527dba62e77 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.014017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.014695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.014973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.015257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.016050 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:27.016358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.017752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.029444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 88/171] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:27.029444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.037764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.038740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.039292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.039797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.040971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] RBAC: Adding query filter params (role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=160bfc163c5d49f9a9bae22a11e7f2e7) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:27.042098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.043899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.054988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 84/172] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:26 2022] GET /identity/v3/role_assignments?user.id=160bfc163c5d49f9a9bae22a11e7f2e7&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:27.056299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.141795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf18f1b-6097-4322-8d05-a36ca50305cf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.165348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36e425-c657-406a-8764-c01d4ddd4ff8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.177700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.178580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.178864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.179144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] PATH_INFO: `/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.180386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] RBAC: Authorizing `identity:get_user(user_id=160bfc163c5d49f9a9bae22a11e7f2e7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.182497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.184286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 89/173] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/users/160bfc163c5d49f9a9bae22a11e7f2e7 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:27.189747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.206647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.207950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.208480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.209059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] PATH_INFO: `/v3/users/6246a26f46e94e7389a45ee846fec1c9` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.210655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] RBAC: Authorizing `identity:get_user(user_id=6246a26f46e94e7389a45ee846fec1c9)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.212806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.215849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 85/174] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/users/6246a26f46e94e7389a45ee846fec1c9 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:27.221649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.299415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a47c21-2bc5-4ae7-8272-79e286c6d6ce None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.330279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2f623-615a-4537-9f4b-f99132baa3c5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.337480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.338247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.338533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.338949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.372055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.373097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.373579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.374095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.417544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.419166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.450415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.452367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.555122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 86/175] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:55:27.561595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.567363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 90/176] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:55:27.576630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.662697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fcccb-d90b-4be9-8a96-2e304b74ae9d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.697072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.698878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.699388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.700424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.701862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.703510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.705449 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 87/177] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:27.710174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3119726-7f2d-42ae-9621-8bac955df8b8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.711885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.772388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.773271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.773554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.773831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] PATH_INFO: `/v3/projects/f510cfe1ac764712bf859c01884e55f7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.775079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] RBAC: Authorizing `identity:get_project(project_id=f510cfe1ac764712bf859c01884e55f7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.776645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.778679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 91/178] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/projects/f510cfe1ac764712bf859c01884e55f7 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:27.784323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:27.818720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dd65e-bd76-4437-9f2a-00a3950d3c24 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.856049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764b19d-501f-40a0-b70b-df164b278741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.857083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764b19d-501f-40a0-b70b-df164b278741 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.857626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764b19d-501f-40a0-b70b-df164b278741 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.858120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764b19d-501f-40a0-b70b-df164b278741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.859240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764b19d-501f-40a0-b70b-df164b278741 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=6246a26f46e94e7389a45ee846fec1c9) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:27.859776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764b19d-501f-40a0-b70b-df164b278741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.861446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764b19d-501f-40a0-b70b-df164b278741 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.868114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 88/179] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/role_assignments?user.id=6246a26f46e94e7389a45ee846fec1c9&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:27.917317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74d757-13d5-4b7f-9d35-bcfc6d2bf2b8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:27.976865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9eedb-38a9-4af9-af51-9f3f72e78095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:27.977538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9eedb-38a9-4af9-af51-9f3f72e78095 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:27.977821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9eedb-38a9-4af9-af51-9f3f72e78095 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:27.978340 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9eedb-38a9-4af9-af51-9f3f72e78095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:27.979339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9eedb-38a9-4af9-af51-9f3f72e78095 None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=f510cfe1ac764712bf859c01884e55f7, user.id=160bfc163c5d49f9a9bae22a11e7f2e7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:27.979930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9eedb-38a9-4af9-af51-9f3f72e78095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:27.981428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9eedb-38a9-4af9-af51-9f3f72e78095 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:27.992605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 92/180] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:27 2022] GET /identity/v3/role_assignments?user.id=160bfc163c5d49f9a9bae22a11e7f2e7&scope.project.id=f510cfe1ac764712bf859c01884e55f7&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:28.641586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fb620-a59e-4b1a-b342-581bb5a82ec2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:28.642087 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fb620-a59e-4b1a-b342-581bb5a82ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:28.642193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fb620-a59e-4b1a-b342-581bb5a82ec2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:28.643663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 89/181] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:28.649140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8589b1-1f6f-439d-ac05-78652be3be0d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:28.649646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8589b1-1f6f-439d-ac05-78652be3be0d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:28.650118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8589b1-1f6f-439d-ac05-78652be3be0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:28.668058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-eb8589b1-1f6f-439d-ac05-78652be3be0d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:28.669527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-eb8589b1-1f6f-439d-ac05-78652be3be0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:28.740532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 93/182] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:28.764596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fb620-a59e-4b1a-b342-581bb5a82ec2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:28.871183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fb620-a59e-4b1a-b342-581bb5a82ec2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:28.907100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:28.907852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:28.908136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:28.908411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:28.919299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:28.927117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:28.928056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:28.928848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 90/183] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:28.934936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8589b1-1f6f-439d-ac05-78652be3be0d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.036693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8589b1-1f6f-439d-ac05-78652be3be0d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.077542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e19b67-f753-4d10-9c81-9cc8343447dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.078227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e19b67-f753-4d10-9c81-9cc8343447dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.078508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e19b67-f753-4d10-9c81-9cc8343447dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.078784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e19b67-f753-4d10-9c81-9cc8343447dc None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.079530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e19b67-f753-4d10-9c81-9cc8343447dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:29.079864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e19b67-f753-4d10-9c81-9cc8343447dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:29.081253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e19b67-f753-4d10-9c81-9cc8343447dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:29.090553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 94/184] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:29.096299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.200619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0251476-ba10-4601-8696-e49bba5f41ad None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.228948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17a703-b7a4-4a0f-8929-9928af0b2b8a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.229247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17a703-b7a4-4a0f-8929-9928af0b2b8a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.229841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17a703-b7a4-4a0f-8929-9928af0b2b8a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.231393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 95/185] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:29.237374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1750f0-9e48-498b-b2c1-d17ecbc0550d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.237755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1750f0-9e48-498b-b2c1-d17ecbc0550d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.238038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1750f0-9e48-498b-b2c1-d17ecbc0550d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.240077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.240955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.241238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.241513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] PATH_INFO: `/v3/users/12b32173f72043d684a71c7411693336` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.242716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] RBAC: Authorizing `identity:get_user(user_id=12b32173f72043d684a71c7411693336)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:29.244509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:29.246062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 91/186] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/users/12b32173f72043d684a71c7411693336 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:29.252063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.258319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cd1750f0-9e48-498b-b2c1-d17ecbc0550d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:29.259503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cd1750f0-9e48-498b-b2c1-d17ecbc0550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:29.328353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 96/187] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:29.357702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1750f0-9e48-498b-b2c1-d17ecbc0550d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.383469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c24e3-a9ff-413d-847a-c63bf4f0fe2d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.425202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.426129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.426413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.426696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.473534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1750f0-9e48-498b-b2c1-d17ecbc0550d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.504922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:29.506508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:29.508830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.509703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.510637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.511111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.521958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:29.530518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:29.531394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:29.532199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:29.536538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 97/188] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:29.537946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.609044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 92/189] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:29.615108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.645180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41044d53-2ad4-4345-9677-59e18621febe None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.680264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.680936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.681216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.681489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.682448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:29.682758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:29.684253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:29.692605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 98/190] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:29.698738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.716254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b203ee8-f218-4a4d-919c-db1a37acdc6a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.751852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.752667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.752948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.753224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.754428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:29.755933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:29.757668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 93/191] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:29.764512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.818663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0364e-47ef-421e-9562-69d44f353fa8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.867965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.869177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.869527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.869823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] PATH_INFO: `/v3/users/49059aebb9d44266b4b8f4984da2e68c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.889657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] RBAC: Authorizing `identity:get_user(user_id=49059aebb9d44266b4b8f4984da2e68c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:29.892180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:29.894480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 99/192] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/users/49059aebb9d44266b4b8f4984da2e68c => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:29.897795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c30fa-fa10-4937-9579-fa7b5c3cb5a4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:29.899829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:29.936641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506871a-ea49-4b7f-a97f-6a9f54edd44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:29.937768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506871a-ea49-4b7f-a97f-6a9f54edd44b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.938250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506871a-ea49-4b7f-a97f-6a9f54edd44b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.938728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506871a-ea49-4b7f-a97f-6a9f54edd44b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.939793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506871a-ea49-4b7f-a97f-6a9f54edd44b None admin] RBAC: Adding query filter params (role.id=1a29288217364c61b38b69e4e823a4d5, scope.project.id=712891263b4f468fb3ce3dda3aeaed04, user.id=12b32173f72043d684a71c7411693336) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:29.940322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506871a-ea49-4b7f-a97f-6a9f54edd44b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:29.942091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506871a-ea49-4b7f-a97f-6a9f54edd44b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:29.955771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 94/193] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/role_assignments?user.id=12b32173f72043d684a71c7411693336&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=1a29288217364c61b38b69e4e823a4d5 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:29.971216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7b3db-6c4f-4a50-8d8b-3a16c020dac7 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.971517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7b3db-6c4f-4a50-8d8b-3a16c020dac7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.971802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7b3db-6c4f-4a50-8d8b-3a16c020dac7 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.973489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 95/194] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:29.979769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587f6a6-16a4-42b7-91f7-e5ea85bb4897 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:29.980065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587f6a6-16a4-42b7-91f7-e5ea85bb4897 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:29.980322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587f6a6-16a4-42b7-91f7-e5ea85bb4897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:29.998053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8587f6a6-16a4-42b7-91f7-e5ea85bb4897 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:29.999251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8587f6a6-16a4-42b7-91f7-e5ea85bb4897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:30.010930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9918a-4678-4c81-be6a-7f3847b3b730 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.051736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.052916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.053325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.053614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.076738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 96/195] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:30.078445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c218cd-286a-454d-9ec5-e4d26edd4681 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.078962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c218cd-286a-454d-9ec5-e4d26edd4681 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.079296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c218cd-286a-454d-9ec5-e4d26edd4681 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.080960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 97/196] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:30.086406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c76e6-b234-4312-a0c0-4534b874bd3e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.087014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c76e6-b234-4312-a0c0-4534b874bd3e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.087371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c76e6-b234-4312-a0c0-4534b874bd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.106321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-125c76e6-b234-4312-a0c0-4534b874bd3e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:30.107595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-125c76e6-b234-4312-a0c0-4534b874bd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:30.141410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:30.144058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:30.185358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 98/197] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:30.185358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c76e6-b234-4312-a0c0-4534b874bd3e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.258866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 100/198] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:30.261390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.287429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c76e6-b234-4312-a0c0-4534b874bd3e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.319866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.320689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.320968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.321247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.331186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:30.340230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:30.347257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 99/199] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:30.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.363772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a11c6-658a-40e2-a65e-7872e0b9d585 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.395918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.396700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.396979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.397253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.398408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:30.399873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:30.402278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 101/200] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:30.404220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.437781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f07718-6785-4bd5-a00b-3fd7f210be0b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.470875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.471915 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.472199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.472477 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.473673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:30.475132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:30.476879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 100/201] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:30 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:30.482649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.497792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61385ff-0518-4aa6-8b8e-6ab571d40db9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.530729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.531319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.531599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.531900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.532642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:30.532946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:30.534312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:30.542037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 102/202] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:30.542812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.577586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf888e-9f54-4f9c-bfc7-5735124e37e2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.609910 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.610570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.610853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.611129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.611862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:30.613292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:30.636156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dc50e-36f5-4958-8e29-e7aedbc878ca None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.667926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.668600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.668875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.669149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.669977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=49059aebb9d44266b4b8f4984da2e68c) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:30.670282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:30.671738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:30.674989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 101/203] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:55:30 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:30.675896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.677278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 103/204] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:30 2022] GET /identity/v3/role_assignments?user.id=49059aebb9d44266b4b8f4984da2e68c&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:30.794368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d7f78-7a93-415e-b163-a28955ac5866 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.831285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.832227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.832510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.832782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] PATH_INFO: `/v3/users/49059aebb9d44266b4b8f4984da2e68c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:30.834122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] RBAC: Authorizing `identity:get_user(user_id=49059aebb9d44266b4b8f4984da2e68c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:30.835674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:30.837741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 102/205] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:30 2022] GET /identity/v3/users/49059aebb9d44266b4b8f4984da2e68c => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:30.844824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:30.952809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fef9e-ccb1-4eee-8d94-702d150f8bda None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:30.989349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:30.990045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:30.990329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:30.990608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:31.064146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:31.065552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:31.163376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 104/206] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:31.169739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:31.270875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c4337-8f6c-4a8b-a06a-c5e511b68de9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:31.305000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdd252-f90b-4d75-890d-eae7fcb253ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:31.305864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdd252-f90b-4d75-890d-eae7fcb253ed None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:31.306144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdd252-f90b-4d75-890d-eae7fcb253ed None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:31.306417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdd252-f90b-4d75-890d-eae7fcb253ed None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:31.308310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdd252-f90b-4d75-890d-eae7fcb253ed None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:31.309751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdd252-f90b-4d75-890d-eae7fcb253ed None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:31.327453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 103/207] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:31 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:31.327592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:31.417187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c292d6-4e71-4c0f-b4bb-eafbd368a1aa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:31.452027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b20ca-a568-4db9-997a-e18be85325a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:31.452926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b20ca-a568-4db9-997a-e18be85325a4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:31.453370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b20ca-a568-4db9-997a-e18be85325a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:31.453778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b20ca-a568-4db9-997a-e18be85325a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:31.454805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b20ca-a568-4db9-997a-e18be85325a4 None admin] RBAC: Adding query filter params (role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=49059aebb9d44266b4b8f4984da2e68c) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:31.455252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b20ca-a568-4db9-997a-e18be85325a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:31.456948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b20ca-a568-4db9-997a-e18be85325a4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:31.464102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 105/208] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:31 2022] GET /identity/v3/role_assignments?user.id=49059aebb9d44266b4b8f4984da2e68c&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:32.935245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8954b50-3517-4ce1-a6c1-92a122aa5d47 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:32.935799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8954b50-3517-4ce1-a6c1-92a122aa5d47 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:32.935911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8954b50-3517-4ce1-a6c1-92a122aa5d47 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:32.937207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 104/209] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:32.943173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af885f-c9b4-48db-88f4-e83735c9b990 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:32.943713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af885f-c9b4-48db-88f4-e83735c9b990 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:32.943947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af885f-c9b4-48db-88f4-e83735c9b990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:32.961553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-97af885f-c9b4-48db-88f4-e83735c9b990 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:32.963001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-97af885f-c9b4-48db-88f4-e83735c9b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:33.035355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 106/210] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:33.058470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8954b50-3517-4ce1-a6c1-92a122aa5d47 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.121900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f40fd-cae5-4676-bffd-c21abeed01a2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.122085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f40fd-cae5-4676-bffd-c21abeed01a2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.122360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f40fd-cae5-4676-bffd-c21abeed01a2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.123740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 107/211] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:33.128661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaaf1b-2ce9-437a-8ccc-8af92c10f8e3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.128942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaaf1b-2ce9-437a-8ccc-8af92c10f8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.129633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaaf1b-2ce9-437a-8ccc-8af92c10f8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.145961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5acaaf1b-2ce9-437a-8ccc-8af92c10f8e3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:33.147081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5acaaf1b-2ce9-437a-8ccc-8af92c10f8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:33.165440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8954b50-3517-4ce1-a6c1-92a122aa5d47 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.202782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.203572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.203880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.204148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.214768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:33.217054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 108/212] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:33.218284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045788-6679-4018-bfe0-ef32386a1f93 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.218602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045788-6679-4018-bfe0-ef32386a1f93 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.218878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3045788-6679-4018-bfe0-ef32386a1f93 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.220296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 109/213] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:33.225288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358a18b-ad10-4e88-b773-c105fd1a80b6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.225576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358a18b-ad10-4e88-b773-c105fd1a80b6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.225853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358a18b-ad10-4e88-b773-c105fd1a80b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:33.229343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:33.230223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:33.231030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 105/214] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:33.236103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.249486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4358a18b-ad10-4e88-b773-c105fd1a80b6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:33.250649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4358a18b-ad10-4e88-b773-c105fd1a80b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:33.323916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 110/215] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:33.324837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358a18b-ad10-4e88-b773-c105fd1a80b6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.337553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14b840-ef14-4936-b4a2-540acc8f13b9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.375324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.376294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.376577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.377020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.377755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:33.378054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:33.379424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:33.388119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 106/216] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:33.393387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.429355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358a18b-ad10-4e88-b773-c105fd1a80b6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.465098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.465951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.466234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.466503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.478472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:33.489038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:33.489925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:33.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:33.491332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 111/217] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:33.491927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.497062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f99125-8695-4263-9a37-c5cd2da5a690 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.534438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.535250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.535536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.535841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.537065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:33.538611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:33.545610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 107/218] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:33.546388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.594001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b343805-b6c7-47b4-9b0e-edd1fde85902 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.626753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.627533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.627862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.628144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] PATH_INFO: `/v3/users/49059aebb9d44266b4b8f4984da2e68c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.629328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] RBAC: Authorizing `identity:get_user(user_id=49059aebb9d44266b4b8f4984da2e68c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:33.630798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:33.637147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 112/219] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/users/49059aebb9d44266b4b8f4984da2e68c => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:33.637147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.650469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb04b8d-4500-44b9-84a2-3cd32cc9faa4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.684265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.685031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.685386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.685734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.686553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:33.686862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:33.688266 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:33.696277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 108/220] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:33.701896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.733379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533b9f6-ef47-492f-9e92-8d5c5ec27625 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.764959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a98f5d-29b5-44d1-b8f8-34527ffdb6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.765620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a98f5d-29b5-44d1-b8f8-34527ffdb6ea None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.765955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a98f5d-29b5-44d1-b8f8-34527ffdb6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.766232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a98f5d-29b5-44d1-b8f8-34527ffdb6ea None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.766932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a98f5d-29b5-44d1-b8f8-34527ffdb6ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:33.768399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a98f5d-29b5-44d1-b8f8-34527ffdb6ea None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:33.796134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da728343-d974-46af-89cb-af70840f5cd4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.827775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.828617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.828899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.829168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.845619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:33.845619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:33.845619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 109/221] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:33.845619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.927247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf1579-ba1f-485f-9903-27b250080a50 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:33.942298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 113/222] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:55:33 2022] POST /identity/v3/users => generated 297 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:33.943602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c740f4-4b88-4a2d-bd33-425f06ff2982 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.944185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c740f4-4b88-4a2d-bd33-425f06ff2982 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.944185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c740f4-4b88-4a2d-bd33-425f06ff2982 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.950964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 114/223] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:33.950964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c740f4-4b88-4a2d-bd33-425f06ff2982 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:33.961793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d367c-7bd3-4b09-8533-101bf2a7790b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:33.962959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d367c-7bd3-4b09-8533-101bf2a7790b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.963373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d367c-7bd3-4b09-8533-101bf2a7790b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.963708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d367c-7bd3-4b09-8533-101bf2a7790b None admin] PATH_INFO: `/v3/users/9f214c7eddfd4c73aeaa731352b5e45b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:33.978845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d367c-7bd3-4b09-8533-101bf2a7790b None admin] RBAC: Authorizing `identity:get_user(user_id=9f214c7eddfd4c73aeaa731352b5e45b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:33.980580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d367c-7bd3-4b09-8533-101bf2a7790b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:33.982671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 110/224] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:33 2022] GET /identity/v3/users/9f214c7eddfd4c73aeaa731352b5e45b => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:33.983703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf067f-8d7e-454c-9086-a34c36eb5c40 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:33.983988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf067f-8d7e-454c-9086-a34c36eb5c40 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:33.984262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf067f-8d7e-454c-9086-a34c36eb5c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.002015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5bdf067f-8d7e-454c-9086-a34c36eb5c40 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:34.003254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5bdf067f-8d7e-454c-9086-a34c36eb5c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:34.061547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c740f4-4b88-4a2d-bd33-425f06ff2982 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.073357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 111/225] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:34.074584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf067f-8d7e-454c-9086-a34c36eb5c40 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:34.098172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.099073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.099361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.099623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/49059aebb9d44266b4b8f4984da2e68c/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.101764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad1577623e54cb486ce87b0fb9cef96, user_id=49059aebb9d44266b4b8f4984da2e68c, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:34.103242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:34.176842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf067f-8d7e-454c-9086-a34c36eb5c40 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.200499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 115/226] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:33 2022] PUT /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/49059aebb9d44266b4b8f4984da2e68c/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:34.201569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:34.222240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.222924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.223213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.223488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.301260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:34.303138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:34.337446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5577e-9193-4ce2-818c-d45299cd5481 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.377871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.378694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.378974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.379250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.392266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:34.401113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:34.402086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:34.403289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 116/227] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:34.410730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:34.433047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 112/228] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:34.439483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:34.524557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee601-5a66-4293-8783-a61df8d335ae None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.551326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab365b-4ffe-4697-ba44-8518806dc155 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.560962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.561640 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.561923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.562198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.562952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:34.563253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:34.564671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:34.573569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 117/229] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:34.578995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:34.586762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.587928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.588211 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.589153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.590486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:34.591977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:34.593751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 113/230] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:34 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:34.599329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:34.702535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9af4d2-b03c-4e15-a4c8-c9b94231720c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.730218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788de316-9ff5-4275-80f7-3ef482a04e90 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.736789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.737451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.737733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.738011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.738852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=9f214c7eddfd4c73aeaa731352b5e45b) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:34.739163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:34.740661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:34.747052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 114/231] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:34 2022] GET /identity/v3/role_assignments?user.id=9f214c7eddfd4c73aeaa731352b5e45b&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:34.766558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.767472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.767781 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.768054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] PATH_INFO: `/v3/users/49059aebb9d44266b4b8f4984da2e68c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.769377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] RBAC: Authorizing `identity:get_user(user_id=49059aebb9d44266b4b8f4984da2e68c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:34.770865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:34.772639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 118/232] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:34 2022] GET /identity/v3/users/49059aebb9d44266b4b8f4984da2e68c => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:34.778598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:34.931420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599610-ed3d-4452-a8a3-5fa2f8816371 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:34.966139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1e7c1-9747-4a4b-b575-32cd48d7af69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:34.966999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1e7c1-9747-4a4b-b575-32cd48d7af69 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:34.967278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1e7c1-9747-4a4b-b575-32cd48d7af69 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:34.967544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1e7c1-9747-4a4b-b575-32cd48d7af69 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:34.968767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1e7c1-9747-4a4b-b575-32cd48d7af69 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:34.970199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1e7c1-9747-4a4b-b575-32cd48d7af69 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:34.972133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 115/233] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:34 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:34.977511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:35.078240 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff629b0-f7c8-468c-a074-1269bda47e3c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:35.112288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d7505-dd00-440a-b421-e603a1f69d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:35.113193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7505-dd00-440a-b421-e603a1f69d3b None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:35.113468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7505-dd00-440a-b421-e603a1f69d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:35.113737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7505-dd00-440a-b421-e603a1f69d3b None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/49059aebb9d44266b4b8f4984da2e68c/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:35.116016 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d7505-dd00-440a-b421-e603a1f69d3b None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad1577623e54cb486ce87b0fb9cef96, user_id=49059aebb9d44266b4b8f4984da2e68c, role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:35.117458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d7505-dd00-440a-b421-e603a1f69d3b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:35.247395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 119/234] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:34 2022] PUT /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/49059aebb9d44266b4b8f4984da2e68c/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:36.897173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374af0a4-d9e1-488e-8951-5add63ec1faf None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:36.897852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374af0a4-d9e1-488e-8951-5add63ec1faf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:36.897852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374af0a4-d9e1-488e-8951-5add63ec1faf None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:36.899349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 116/235] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:36.905290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a1df7-e758-429c-bd94-6f83bd7c3c21 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:36.905826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a1df7-e758-429c-bd94-6f83bd7c3c21 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:36.905826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a1df7-e758-429c-bd94-6f83bd7c3c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:36.928966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-af8a1df7-e758-429c-bd94-6f83bd7c3c21 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:36.930213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-af8a1df7-e758-429c-bd94-6f83bd7c3c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:37.012597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 120/236] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:37.013673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff6685-e0a4-4efb-9815-0fa204849543 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.013961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff6685-e0a4-4efb-9815-0fa204849543 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.014237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff6685-e0a4-4efb-9815-0fa204849543 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.015998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 117/237] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:37.021114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c8e5-a9eb-44e8-be73-081e87c141f9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.021397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c8e5-a9eb-44e8-be73-081e87c141f9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.021665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c8e5-a9eb-44e8-be73-081e87c141f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.039999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff6685-e0a4-4efb-9815-0fa204849543 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.042329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ef31c8e5-a9eb-44e8-be73-081e87c141f9 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:37.043673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ef31c8e5-a9eb-44e8-be73-081e87c141f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:37.110068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 121/238] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:37.111129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bbe94-bb2d-47bf-9a0e-1ef88673c4c7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.111410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bbe94-bb2d-47bf-9a0e-1ef88673c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.111737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bbe94-bb2d-47bf-9a0e-1ef88673c4c7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.113070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 122/239] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:37.125243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cf03b-53a7-4672-ad50-c983d6e1f12e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.125513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cf03b-53a7-4672-ad50-c983d6e1f12e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.125790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cf03b-53a7-4672-ad50-c983d6e1f12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.146334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a50cf03b-53a7-4672-ad50-c983d6e1f12e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:37.147526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a50cf03b-53a7-4672-ad50-c983d6e1f12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:37.158291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff6685-e0a4-4efb-9815-0fa204849543 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.191464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.192280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.192559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.192828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.203700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:37.211399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:37.212647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 118/240] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:37.214253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.222919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 123/241] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:37.224184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cf03b-53a7-4672-ad50-c983d6e1f12e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.316454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45962a-8045-41e0-b6e0-c6556712690a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.349674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cf03b-53a7-4672-ad50-c983d6e1f12e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.350951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.351842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.352128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.352400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.363276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:37.370630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:37.371510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:37.372488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 119/242] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:37.373142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.391228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.392223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.392645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.393058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.393943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:37.394387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:37.396142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:37.405601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 124/243] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:37.406729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.477217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444bcfb-5c32-4feb-9980-f1b3f07b66de None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.508236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808b6bf-ba1a-4dcc-ace8-6bfb1f86a02d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.512411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.513206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.513488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.513763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.515006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:37.516480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:37.522655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 120/244] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:37.522655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.547232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32add86b-713c-4ce4-8c2f-c8e15777e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.547930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32add86b-713c-4ce4-8c2f-c8e15777e605 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.548214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32add86b-713c-4ce4-8c2f-c8e15777e605 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.548486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32add86b-713c-4ce4-8c2f-c8e15777e605 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.549230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32add86b-713c-4ce4-8c2f-c8e15777e605 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:37.549529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32add86b-713c-4ce4-8c2f-c8e15777e605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:37.550911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32add86b-713c-4ce4-8c2f-c8e15777e605 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:37.559941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 125/245] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:37.564607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96153798-0d90-406e-b4aa-1b698e7d82ef None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.564607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96153798-0d90-406e-b4aa-1b698e7d82ef None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.564607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96153798-0d90-406e-b4aa-1b698e7d82ef None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.564607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 126/246] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:37.564607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96153798-0d90-406e-b4aa-1b698e7d82ef None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.619644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaa3b2-edee-41a9-b29c-1b7fbba038bb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.653747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.654640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.654918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.655208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] PATH_INFO: `/v3/users/49059aebb9d44266b4b8f4984da2e68c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.656423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] RBAC: Authorizing `identity:get_user(user_id=49059aebb9d44266b4b8f4984da2e68c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:37.657953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:37.659574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 121/247] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/users/49059aebb9d44266b4b8f4984da2e68c => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:37.660347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.670180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96153798-0d90-406e-b4aa-1b698e7d82ef None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.704297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7db30-903f-47f9-9796-e7c1bf997c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.704968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7db30-903f-47f9-9796-e7c1bf997c4b None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.705246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7db30-903f-47f9-9796-e7c1bf997c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.705518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7db30-903f-47f9-9796-e7c1bf997c4b None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.706221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7db30-903f-47f9-9796-e7c1bf997c4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:37.707666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7db30-903f-47f9-9796-e7c1bf997c4b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:37.758191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105add4-5d3c-45f2-8936-4964749d2d3d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.789887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.790749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.791033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.791308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] PATH_INFO: `/v3/users/64665d9cdb8c4e579a45963e8d4ddae3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.805100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 127/248] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:55:37 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:37.805259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] RBAC: Authorizing `identity:get_user(user_id=64665d9cdb8c4e579a45963e8d4ddae3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:37.806416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:37.808278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 122/249] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/users/64665d9cdb8c4e579a45963e8d4ddae3 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:37.813670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:37.814227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65632e0-6c0b-4664-89c0-41f2a36b2d23 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.814581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65632e0-6c0b-4664-89c0-41f2a36b2d23 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.814855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65632e0-6c0b-4664-89c0-41f2a36b2d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.864813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b65632e0-6c0b-4664-89c0-41f2a36b2d23 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:37.866026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b65632e0-6c0b-4664-89c0-41f2a36b2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:37.920826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb46993-b899-4782-bcf2-d323b4b255c0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:37.953347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 128/250] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:37.959050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:37.959744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:37.960029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:37.960301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:37.960545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65632e0-6c0b-4664-89c0-41f2a36b2d23 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.038837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:38.040372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:38.086388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65632e0-6c0b-4664-89c0-41f2a36b2d23 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:38.133959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c1d4c-52af-436e-95e9-f393c3035bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:38.134613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c1d4c-52af-436e-95e9-f393c3035bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.134893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c1d4c-52af-436e-95e9-f393c3035bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.135161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c1d4c-52af-436e-95e9-f393c3035bb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.147532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 123/251] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:38.148708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.224731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c1d4c-52af-436e-95e9-f393c3035bb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:38.226225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c1d4c-52af-436e-95e9-f393c3035bb5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:38.252271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ab311-19b6-41d9-b793-c28c75057dd0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:38.291217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:38.293668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.293668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.293668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.305131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:38.314294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:38.315548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 124/252] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:38.316534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.355567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 129/253] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:38.357193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6c0a4-20f6-4d8a-805c-1d6e5b85a708 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.358078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6c0a4-20f6-4d8a-805c-1d6e5b85a708 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.358581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6c0a4-20f6-4d8a-805c-1d6e5b85a708 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.364180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 130/254] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:38.365867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6c0a4-20f6-4d8a-805c-1d6e5b85a708 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.423544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae497be4-6318-4e90-b1f5-8d97c02855ef None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:38.459376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:38.460461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.460844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.461399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.463124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:38.465371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:38.471593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 125/255] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:38.471593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.505461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6c0a4-20f6-4d8a-805c-1d6e5b85a708 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:38.554411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf879cc-3646-4594-b423-2ecfa0e7762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:38.555608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf879cc-3646-4594-b423-2ecfa0e7762f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.556549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf879cc-3646-4594-b423-2ecfa0e7762f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.557080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf879cc-3646-4594-b423-2ecfa0e7762f None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.558528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf879cc-3646-4594-b423-2ecfa0e7762f None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:38.559109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf879cc-3646-4594-b423-2ecfa0e7762f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:38.561012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf879cc-3646-4594-b423-2ecfa0e7762f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:38.575661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 131/256] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:38.575661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a4fa0-afd6-4c0f-9605-23089676f10b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.575661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a4fa0-afd6-4c0f-9605-23089676f10b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.575661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a4fa0-afd6-4c0f-9605-23089676f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.580072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad14a-bb7e-456e-97ac-704181816fa0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:38.600613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-653a4fa0-afd6-4c0f-9605-23089676f10b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:38.602603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-653a4fa0-afd6-4c0f-9605-23089676f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:38.615967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:38.616826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.617105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.617429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.618636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:38.620807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:38.622566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 126/257] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:38.623320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.691010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 132/258] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:38.692232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a4fa0-afd6-4c0f-9605-23089676f10b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.755714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65cf9f0-2516-4b75-892f-dffcd006f0c8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:38.788207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:38.788913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.789234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.789452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.790274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=49059aebb9d44266b4b8f4984da2e68c) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:38.790580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:38.792593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:38.798788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 127/259] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/role_assignments?user.id=49059aebb9d44266b4b8f4984da2e68c&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:38.803762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:38.829796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a4fa0-afd6-4c0f-9605-23089676f10b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:38.912658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:38.913612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:38.913887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:38.914202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] PATH_INFO: `/v3/users/9f214c7eddfd4c73aeaa731352b5e45b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:38.915556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] RBAC: Authorizing `identity:get_user(user_id=9f214c7eddfd4c73aeaa731352b5e45b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:38.917756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:38.921622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 133/260] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/users/9f214c7eddfd4c73aeaa731352b5e45b => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:38.922279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:39.020467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190b685-e250-41f5-8897-56830e2d409b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:39.056241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:39.057267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:39.058963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:39.059451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:39.060502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] RBAC: Adding query filter params (role.id=1a29288217364c61b38b69e4e823a4d5, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=64665d9cdb8c4e579a45963e8d4ddae3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:39.061013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:39.062659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:39.068852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 128/261] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/role_assignments?user.id=64665d9cdb8c4e579a45963e8d4ddae3&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=1a29288217364c61b38b69e4e823a4d5 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:39.069872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:39.099590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dee53d-4710-4244-a4c6-1085aed8495a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:39.189092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:39.189632 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0497b16-2317-4383-9268-1aeaf9159106 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:39.189923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:39.190202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:39.190473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:39.224650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:39.231086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:39.232380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:39.232882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:39.233356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:39.234886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:39.236647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:39.237754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:39.238650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:39.239430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 134/262] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:39.241021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 129/263] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:39 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:39.244726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:39.247368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:39.394467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d0ea-f99e-411b-a293-f0613cd35775 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:39.441361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:39.442252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:39.442531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:39.442803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/9f214c7eddfd4c73aeaa731352b5e45b/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:39.445804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad1577623e54cb486ce87b0fb9cef96, user_id=9f214c7eddfd4c73aeaa731352b5e45b, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:39.447258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:39.471005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde24430-44bb-41a4-956d-6205df6fe6db None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:39.506806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:39.507477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:39.507799 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:39.508077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:39.508828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:39.509136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:39.511035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:39.520058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 135/264] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:39.525763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:39.585113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 130/265] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:39 2022] PUT /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/9f214c7eddfd4c73aeaa731352b5e45b/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 14:55:39.639700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef7897-7f00-4435-a70c-ddeed2e6acc0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:39.677573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:39.678380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:39.678661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:39.678935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] PATH_INFO: `/v3/users/49059aebb9d44266b4b8f4984da2e68c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:39.680225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] RBAC: Authorizing `identity:get_user(user_id=49059aebb9d44266b4b8f4984da2e68c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:39.681721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:39.683952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 136/266] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:39 2022] GET /identity/v3/users/49059aebb9d44266b4b8f4984da2e68c => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:39.689639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:39.834055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ee355-855a-4d6a-acdc-0649294c9751 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:39.872656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:39.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:39.873589 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:39.873860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:39.943186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:39.944599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:40.043020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 131/267] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:40.048780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:40.215707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd07add-548d-47df-9f48-4da0d807d426 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:40.262227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82828be-b536-444e-8ae6-94b33c485a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:40.263098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82828be-b536-444e-8ae6-94b33c485a72 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:40.263376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82828be-b536-444e-8ae6-94b33c485a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:40.263668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82828be-b536-444e-8ae6-94b33c485a72 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:40.268337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82828be-b536-444e-8ae6-94b33c485a72 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:40.269802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82828be-b536-444e-8ae6-94b33c485a72 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:40.271715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 137/268] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:40 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:40.277394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:40.490671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afafe48-4be9-4109-9afe-7ca825bac1e1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:40.525573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeafd06-8cd8-499c-ad30-eabc04128981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:40.526278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeafd06-8cd8-499c-ad30-eabc04128981 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:40.526565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeafd06-8cd8-499c-ad30-eabc04128981 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:40.526837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeafd06-8cd8-499c-ad30-eabc04128981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:40.528008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeafd06-8cd8-499c-ad30-eabc04128981 None admin] RBAC: Adding query filter params (role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=49059aebb9d44266b4b8f4984da2e68c) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:40.528319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeafd06-8cd8-499c-ad30-eabc04128981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:40.529823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeafd06-8cd8-499c-ad30-eabc04128981 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:40.537251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 132/269] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:40 2022] GET /identity/v3/role_assignments?user.id=49059aebb9d44266b4b8f4984da2e68c&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:40.738617 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293ff4d-48be-4635-8538-a774cdcb9955 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:40.739055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293ff4d-48be-4635-8538-a774cdcb9955 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:40.739157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293ff4d-48be-4635-8538-a774cdcb9955 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:40.740590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 138/270] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:40.757278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4fea0-fce7-4955-8d34-3881a44e7621 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:40.757557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4fea0-fce7-4955-8d34-3881a44e7621 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:40.757761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4fea0-fce7-4955-8d34-3881a44e7621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:40.777549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6cc4fea0-fce7-4955-8d34-3881a44e7621 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:40.778726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6cc4fea0-fce7-4955-8d34-3881a44e7621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:40.794899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8181c-fc1e-4350-ae14-1003121c0ce0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:40.795191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8181c-fc1e-4350-ae14-1003121c0ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:40.795472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8181c-fc1e-4350-ae14-1003121c0ce0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:40.797107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 139/271] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:40.802014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63d64-0356-46f0-ab06-e41ad0cb1667 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:40.802298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63d64-0356-46f0-ab06-e41ad0cb1667 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:40.802576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63d64-0356-46f0-ab06-e41ad0cb1667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:40.825531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-75d63d64-0356-46f0-ab06-e41ad0cb1667 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:40.826742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-75d63d64-0356-46f0-ab06-e41ad0cb1667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:40.851136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 133/272] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:40.875953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4fea0-fce7-4955-8d34-3881a44e7621 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:40.903321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 140/273] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:40.928429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63d64-0356-46f0-ab06-e41ad0cb1667 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:40.980967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4fea0-fce7-4955-8d34-3881a44e7621 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:41.020964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:41.021684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.021961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.022255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.037175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:41.038230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63d64-0356-46f0-ab06-e41ad0cb1667 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:41.047058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:41.048374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:41.049168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:41.052035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 134/274] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:41.057738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:41.077284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:41.078095 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.078403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.078657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.079865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:41.081309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:41.082936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 141/275] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:41.088709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:41.179461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ba86-fb76-4e9c-857f-537b66687275 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:41.197634 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815e562-898e-404b-ba31-5c0ad7927209 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:41.224116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:41.224786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.225065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.225337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.226080 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:41.226383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:41.227786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:41.236693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 135/276] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:41.241710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:41.242168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:41.242406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.242691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.242966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.243703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:41.246493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:41.340911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c58e-7896-458c-8b8b-ad34bbe41e83 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:41.352654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 142/277] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:55:41 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:41.389147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10b46-82c4-4151-a96a-a240317d3ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:41.390225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10b46-82c4-4151-a96a-a240317d3ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.390512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10b46-82c4-4151-a96a-a240317d3ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.390784 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10b46-82c4-4151-a96a-a240317d3ce8 None admin] PATH_INFO: `/v3/users/a26140679fc045b8a093ff71e118772b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.461468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10b46-82c4-4151-a96a-a240317d3ce8 None admin] RBAC: Authorizing `identity:get_user(user_id=a26140679fc045b8a093ff71e118772b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:41.463262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10b46-82c4-4151-a96a-a240317d3ce8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:41.465698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 136/278] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:41 2022] GET /identity/v3/users/a26140679fc045b8a093ff71e118772b => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:41.472292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:41.582214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a0c9-8fdb-43fc-8a84-fb667b6f567c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:41.616183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65a289-eb90-44e0-941d-fdd720e794c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:41.616777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65a289-eb90-44e0-941d-fdd720e794c5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.617064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65a289-eb90-44e0-941d-fdd720e794c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.617341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65a289-eb90-44e0-941d-fdd720e794c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.618196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65a289-eb90-44e0-941d-fdd720e794c5 None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, user.id=a26140679fc045b8a093ff71e118772b, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:41.618509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65a289-eb90-44e0-941d-fdd720e794c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:41.619991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65a289-eb90-44e0-941d-fdd720e794c5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:41.626296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 143/279] 10.222.0.71 () {56 vars in 1308 bytes} [Mon May 16 14:55:41 2022] GET /identity/v3/role_assignments?user.id=a26140679fc045b8a093ff71e118772b&scope.system=all&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:41.666796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3ea6f-6a2e-4041-9b45-cf275233023b None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.667702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3ea6f-6a2e-4041-9b45-cf275233023b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.668094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3ea6f-6a2e-4041-9b45-cf275233023b None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.669740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 137/280] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:41.675468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785bf84-51fe-44fa-93c2-5107bfe32d8c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.675844 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785bf84-51fe-44fa-93c2-5107bfe32d8c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.676130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785bf84-51fe-44fa-93c2-5107bfe32d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.697529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2785bf84-51fe-44fa-93c2-5107bfe32d8c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:41.699411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2785bf84-51fe-44fa-93c2-5107bfe32d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:41.783424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 144/281] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:41.807326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3ea6f-6a2e-4041-9b45-cf275233023b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:41.935079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3ea6f-6a2e-4041-9b45-cf275233023b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:41.979089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57645cb9-519e-46f2-a5f1-99746568e0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:41.980468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645cb9-519e-46f2-a5f1-99746568e0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:41.980754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645cb9-519e-46f2-a5f1-99746568e0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:41.981019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645cb9-519e-46f2-a5f1-99746568e0e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:41.998041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57645cb9-519e-46f2-a5f1-99746568e0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-57645cb9-519e-46f2-a5f1-99746568e0e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:42.006000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:42.006902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:42.007723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 138/282] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:42.014651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785bf84-51fe-44fa-93c2-5107bfe32d8c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.132576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563952b6-cab9-4ac2-a8cc-3ab78a855e97 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.133522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563952b6-cab9-4ac2-a8cc-3ab78a855e97 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.133522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563952b6-cab9-4ac2-a8cc-3ab78a855e97 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.134607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 139/283] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:42.140327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a45fcb-3153-4664-b8a8-3b7a06b9b0e2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.140564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a45fcb-3153-4664-b8a8-3b7a06b9b0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.140905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a45fcb-3153-4664-b8a8-3b7a06b9b0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.165616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785bf84-51fe-44fa-93c2-5107bfe32d8c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:42.166732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-77a45fcb-3153-4664-b8a8-3b7a06b9b0e2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:42.168834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-77a45fcb-3153-4664-b8a8-3b7a06b9b0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:42.207562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:42.208269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.208556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.209135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.209891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:42.210356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:42.211796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:42.224424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 145/284] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:42.230134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.321917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 140/285] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:42.378826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bde1fc-da27-4132-b6c8-7a0511aae31d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:42.386535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a45fcb-3153-4664-b8a8-3b7a06b9b0e2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.458111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:42.459269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.459746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.460192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] PATH_INFO: `/v3/users/64665d9cdb8c4e579a45963e8d4ddae3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.461702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] RBAC: Authorizing `identity:get_user(user_id=64665d9cdb8c4e579a45963e8d4ddae3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:42.463604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:42.467362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 146/286] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:42 2022] GET /identity/v3/users/64665d9cdb8c4e579a45963e8d4ddae3 => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:42.472676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.542904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a45fcb-3153-4664-b8a8-3b7a06b9b0e2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:42.576073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d4207-0bde-4638-b8bd-3af4ed477d0b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:42.582824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:42.584024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.584024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.584295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.595393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:42.603118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:42.604021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:42.605379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 141/287] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:42.616682 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.634324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:42.635511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.636671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.637121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.640615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:42.642299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:42.644553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 147/288] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:42 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:42.649652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.722498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401f388-8d25-40a3-860c-60f421997207 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:42.759339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:42.759339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.759339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.759339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.759757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:42.759757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:42.759757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:42.768957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 142/289] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:42.770510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c896e-026f-4385-b9d4-09c6b96c1784 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:42.774704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.840204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:42.841486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.841965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.842428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/64665d9cdb8c4e579a45963e8d4ddae3/roles/1a29288217364c61b38b69e4e823a4d5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.846493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad1577623e54cb486ce87b0fb9cef96, user_id=64665d9cdb8c4e579a45963e8d4ddae3, role_id=1a29288217364c61b38b69e4e823a4d5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:42.848264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:42.880472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee33f1-54ff-415e-9a52-6da7277daeac None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:42.914471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:42.915334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:42.915608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:42.915908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] PATH_INFO: `/v3/users/9f214c7eddfd4c73aeaa731352b5e45b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:42.917153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] RBAC: Authorizing `identity:get_user(user_id=9f214c7eddfd4c73aeaa731352b5e45b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:42.918688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:42.924822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 143/290] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:42 2022] GET /identity/v3/users/9f214c7eddfd4c73aeaa731352b5e45b => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:42.925995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:42.955324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 148/291] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:55:42 2022] PUT /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/users/64665d9cdb8c4e579a45963e8d4ddae3/roles/1a29288217364c61b38b69e4e823a4d5 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:43.036759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b665d95-1536-414e-8420-7c6cfe2d85c5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:43.075580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:43.076379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:43.076658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:43.076927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:43.160895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:43.162375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:43.287109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 144/292] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:55:43.292853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:43.414455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95bbbf-bc88-4e10-8d72-c535aac97a28 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:43.456664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf86b5-4f18-49f1-ade1-bd1dc1765c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:43.457522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf86b5-4f18-49f1-ade1-bd1dc1765c94 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:43.457851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf86b5-4f18-49f1-ade1-bd1dc1765c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:43.458131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf86b5-4f18-49f1-ade1-bd1dc1765c94 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:43.459301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf86b5-4f18-49f1-ade1-bd1dc1765c94 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:43.460823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf86b5-4f18-49f1-ade1-bd1dc1765c94 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:43.464575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 149/293] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:43 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:43.469402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:43.577188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f113b-99c6-43ca-999e-ffa1a0097f48 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:43.625183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581d718-3852-460e-9dea-ad9a957b8ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:43.625839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581d718-3852-460e-9dea-ad9a957b8ccb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:43.626120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581d718-3852-460e-9dea-ad9a957b8ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:43.626391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581d718-3852-460e-9dea-ad9a957b8ccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:43.627223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581d718-3852-460e-9dea-ad9a957b8ccb None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=9f214c7eddfd4c73aeaa731352b5e45b) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:43.627527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581d718-3852-460e-9dea-ad9a957b8ccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:43.628984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581d718-3852-460e-9dea-ad9a957b8ccb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:43.640515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 145/294] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:43 2022] GET /identity/v3/role_assignments?user.id=9f214c7eddfd4c73aeaa731352b5e45b&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:44.313492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03e9fd-efb9-47da-b150-ab2394e76982 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.314183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03e9fd-efb9-47da-b150-ab2394e76982 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.314670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03e9fd-efb9-47da-b150-ab2394e76982 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.316465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 150/295] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:44.321921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5126e-2cef-487b-a6be-2b0836cfd306 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.322735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5126e-2cef-487b-a6be-2b0836cfd306 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.324167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5126e-2cef-487b-a6be-2b0836cfd306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.341265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-abe5126e-2cef-487b-a6be-2b0836cfd306 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:44.342739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-abe5126e-2cef-487b-a6be-2b0836cfd306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:44.367565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e70018-65d9-4564-8553-1dbe40f4cf2d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.367937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e70018-65d9-4564-8553-1dbe40f4cf2d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.368150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e70018-65d9-4564-8553-1dbe40f4cf2d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.369685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 151/296] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:44.376763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4e3c6-13aa-4150-883c-35078538cc57 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.377383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4e3c6-13aa-4150-883c-35078538cc57 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.377860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4e3c6-13aa-4150-883c-35078538cc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.396289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6ea4e3c6-13aa-4150-883c-35078538cc57 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:44.397466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6ea4e3c6-13aa-4150-883c-35078538cc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:44.420685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 146/297] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:44.443315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5126e-2cef-487b-a6be-2b0836cfd306 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:44.474358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 152/298] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:44.476392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d1bbc-0d73-4614-86d6-92cd3ca42628 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.476884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d1bbc-0d73-4614-86d6-92cd3ca42628 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.477367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d1bbc-0d73-4614-86d6-92cd3ca42628 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.478821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 153/299] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:44.485191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031edac-e8e3-4d19-ba2a-7487bc2d78b4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.485761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031edac-e8e3-4d19-ba2a-7487bc2d78b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.486235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031edac-e8e3-4d19-ba2a-7487bc2d78b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.504126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4031edac-e8e3-4d19-ba2a-7487bc2d78b4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:44.505298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4031edac-e8e3-4d19-ba2a-7487bc2d78b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:44.551948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5126e-2cef-487b-a6be-2b0836cfd306 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:44.570431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 154/300] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:44.571445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031edac-e8e3-4d19-ba2a-7487bc2d78b4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:44.591953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:44.593508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.593849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.594130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.595766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:44.597678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:44.600959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 147/301] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:44.601832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:44.675697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031edac-e8e3-4d19-ba2a-7487bc2d78b4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:44.699357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38a80e-210a-488d-87be-272fbd3b7137 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:44.708321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:44.709115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.709395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.709665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.720409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:44.728013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:44.728908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:44.729797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 155/302] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:44.730660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:44.737925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:44.738719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.739001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.739270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.749466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:44.757066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:44.758100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:44.759010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 148/303] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:44.764258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:44.828432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c4478-fed5-429a-9cb4-d09f4dd44187 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:44.864954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d9ee5-238a-473d-b807-0527ff6a1db1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:44.865890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:44.866592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.866873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.867145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] PATH_INFO: `/v3/groups` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.868115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:44.869313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:44.900812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:44.907100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:44.907100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:44.907100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:44.907100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:44.907100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:44.907100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:44.912315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 149/304] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:44.913089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:44.960250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 156/305] 10.222.0.71 () {60 vars in 1134 bytes} [Mon May 16 14:55:44 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:44.960250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:45.042734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c6aa-f541-4076-b34b-57cfb29dee0c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:45.080644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:45.081291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.081574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.081847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.082609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:45.082912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:45.084359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:45.098656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56384a-0226-401c-aa72-dc209bfbd08b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:45.098959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 150/306] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:45.100370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:45.144666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:45.146756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.146756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.146756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] PATH_INFO: `/v3/users/a26140679fc045b8a093ff71e118772b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.149195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] RBAC: Authorizing `identity:get_user(user_id=a26140679fc045b8a093ff71e118772b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:45.150109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:45.152661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 157/307] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:44 2022] GET /identity/v3/users/a26140679fc045b8a093ff71e118772b => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:45.159022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:45.221369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef6463-bf20-4da2-9672-0c6e9f4516e4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:45.261167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:45.262560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.263159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.263586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] PATH_INFO: `/v3/users/9dc57f8c9e8e4235a47e31fce2ea8701` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.280878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] RBAC: Authorizing `identity:get_user(user_id=9dc57f8c9e8e4235a47e31fce2ea8701)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:45.281514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f43e11-9f13-40d3-8365-ae94267ece52 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:45.282749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:45.285899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 151/308] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:45 2022] GET /identity/v3/users/9dc57f8c9e8e4235a47e31fce2ea8701 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:45.291843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:45.326335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8fc5a-c61e-42bb-8edd-6577bc49aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:45.327360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8fc5a-c61e-42bb-8edd-6577bc49aae3 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.327936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8fc5a-c61e-42bb-8edd-6577bc49aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.328311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8fc5a-c61e-42bb-8edd-6577bc49aae3 None admin] PATH_INFO: `/v3/system/users/a26140679fc045b8a093ff71e118772b/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.330479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8fc5a-c61e-42bb-8edd-6577bc49aae3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a26140679fc045b8a093ff71e118772b, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:45.332349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8fc5a-c61e-42bb-8edd-6577bc49aae3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:45.405059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fdd70-2118-40e3-8efc-aab0ace4a503 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:45.414333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 158/309] 10.222.0.71 () {58 vars in 1334 bytes} [Mon May 16 14:55:45 2022] PUT /identity/v3/system/users/a26140679fc045b8a093ff71e118772b/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:45.450061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e19686-46eb-42fc-a60a-bc6e4623da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:45.451051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e19686-46eb-42fc-a60a-bc6e4623da45 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.451588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e19686-46eb-42fc-a60a-bc6e4623da45 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.452085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e19686-46eb-42fc-a60a-bc6e4623da45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.453246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e19686-46eb-42fc-a60a-bc6e4623da45 None admin] RBAC: Adding query filter params (role.id=1a29288217364c61b38b69e4e823a4d5, user.id=9dc57f8c9e8e4235a47e31fce2ea8701, scope.system=all) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:45.453736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e19686-46eb-42fc-a60a-bc6e4623da45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:45.455481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e19686-46eb-42fc-a60a-bc6e4623da45 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:45.462213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 152/310] 10.222.0.71 () {56 vars in 1308 bytes} [Mon May 16 14:55:45 2022] GET /identity/v3/role_assignments?user.id=9dc57f8c9e8e4235a47e31fce2ea8701&scope.system=all&role.id=1a29288217364c61b38b69e4e823a4d5 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:45.554100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a6001-dfcc-4a49-8dde-ba32002affe8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.554621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a6001-dfcc-4a49-8dde-ba32002affe8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.554720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a6001-dfcc-4a49-8dde-ba32002affe8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.556398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 159/311] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:45.563293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42669-df59-4463-b754-2bb073d022a1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.563584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42669-df59-4463-b754-2bb073d022a1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.563909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42669-df59-4463-b754-2bb073d022a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.583140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-56e42669-df59-4463-b754-2bb073d022a1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:45.584349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-56e42669-df59-4463-b754-2bb073d022a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:45.663440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 153/312] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:45.687411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a6001-dfcc-4a49-8dde-ba32002affe8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:45.791464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a6001-dfcc-4a49-8dde-ba32002affe8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:45.827019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:45.828186 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:45.828670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:45.829126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:45.846496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:45.854481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:45.856032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:45.856898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:45.858939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 160/313] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:45.864962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42669-df59-4463-b754-2bb073d022a1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:45.967605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42669-df59-4463-b754-2bb073d022a1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:46.002268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873933c2-140a-420f-b540-aede114af027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:46.003043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873933c2-140a-420f-b540-aede114af027 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:46.003326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873933c2-140a-420f-b540-aede114af027 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:46.003652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873933c2-140a-420f-b540-aede114af027 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:46.004410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873933c2-140a-420f-b540-aede114af027 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:46.004715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873933c2-140a-420f-b540-aede114af027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:46.006142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873933c2-140a-420f-b540-aede114af027 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:46.015761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 154/314] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:46.021233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:46.122809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e5bad-9466-4314-9940-9bb20897a63c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:46.157102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:46.158058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:46.158350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:46.158626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] PATH_INFO: `/v3/users/64665d9cdb8c4e579a45963e8d4ddae3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:46.160001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] RBAC: Authorizing `identity:get_user(user_id=64665d9cdb8c4e579a45963e8d4ddae3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:46.161505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:46.163255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 161/315] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:46 2022] GET /identity/v3/users/64665d9cdb8c4e579a45963e8d4ddae3 => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:46.169799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873933c2-140a-420f-b540-aede114af027 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:46.272762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873933c2-140a-420f-b540-aede114af027 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:46.307857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:46.308815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:46.309224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:46.309507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:46.378737 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:46.380142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:46.482421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 155/316] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:46.488046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:46.593728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b183d9-53d6-4254-b3ef-5308829b3519 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:46.629733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a3ab3-7cd0-4617-9d31-90cb13269b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:46.630512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a3ab3-7cd0-4617-9d31-90cb13269b4f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:46.630789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a3ab3-7cd0-4617-9d31-90cb13269b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:46.631069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a3ab3-7cd0-4617-9d31-90cb13269b4f None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:46.632272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a3ab3-7cd0-4617-9d31-90cb13269b4f None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:46.633720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a3ab3-7cd0-4617-9d31-90cb13269b4f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:46.635709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 162/317] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:46 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:46.642120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:46.741992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f409c7-bd20-440a-a5d2-80dcea773eba None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:46.780696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff856f2-79d4-47e8-9e74-0585534b6843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:46.781686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff856f2-79d4-47e8-9e74-0585534b6843 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:46.782146 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff856f2-79d4-47e8-9e74-0585534b6843 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:46.782591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff856f2-79d4-47e8-9e74-0585534b6843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:46.783610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff856f2-79d4-47e8-9e74-0585534b6843 None admin] RBAC: Adding query filter params (role.id=1a29288217364c61b38b69e4e823a4d5, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96, user.id=64665d9cdb8c4e579a45963e8d4ddae3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:46.784124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff856f2-79d4-47e8-9e74-0585534b6843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:46.785743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff856f2-79d4-47e8-9e74-0585534b6843 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:46.792524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 156/318] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:55:46 2022] GET /identity/v3/role_assignments?user.id=64665d9cdb8c4e579a45963e8d4ddae3&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=1a29288217364c61b38b69e4e823a4d5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:47.322336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5a9f8-9df1-40ed-bf9b-f682f596d7b7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.322620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5a9f8-9df1-40ed-bf9b-f682f596d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.322893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5a9f8-9df1-40ed-bf9b-f682f596d7b7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.324602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 163/319] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:47.330179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1c2b-c97a-4623-9bd3-422632b6ea43 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.330469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1c2b-c97a-4623-9bd3-422632b6ea43 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.330744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1c2b-c97a-4623-9bd3-422632b6ea43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.349096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-951d1c2b-c97a-4623-9bd3-422632b6ea43 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:47.350276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-951d1c2b-c97a-4623-9bd3-422632b6ea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:47.415879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 157/320] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:47.438526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5a9f8-9df1-40ed-bf9b-f682f596d7b7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:47.537235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5a9f8-9df1-40ed-bf9b-f682f596d7b7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:47.573794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285621d-ab9e-4d46-ac27-8850e44e70c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:47.574494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285621d-ab9e-4d46-ac27-8850e44e70c0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.574780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285621d-ab9e-4d46-ac27-8850e44e70c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.575057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285621d-ab9e-4d46-ac27-8850e44e70c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.576250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285621d-ab9e-4d46-ac27-8850e44e70c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:47.577716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285621d-ab9e-4d46-ac27-8850e44e70c0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:47.579504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 164/321] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:55:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:47.584889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d1c2b-c97a-4623-9bd3-422632b6ea43 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:47.687276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d1c2b-c97a-4623-9bd3-422632b6ea43 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:47.720749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf52186-2d88-44de-9905-b673a2dece2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:47.721413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52186-2d88-44de-9905-b673a2dece2e None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.721695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52186-2d88-44de-9905-b673a2dece2e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.721968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52186-2d88-44de-9905-b673a2dece2e None admin] PATH_INFO: `/v3/groups` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.722669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf52186-2d88-44de-9905-b673a2dece2e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:47.724135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf52186-2d88-44de-9905-b673a2dece2e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:47.736694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c55c-c548-4b9f-a478-3033efc1d31b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.737197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c55c-c548-4b9f-a478-3033efc1d31b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.737197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c55c-c548-4b9f-a478-3033efc1d31b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.738415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 165/322] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:47.743205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc402c74-8df3-4a66-94e6-256f4d51d570 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.743494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc402c74-8df3-4a66-94e6-256f4d51d570 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.743811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc402c74-8df3-4a66-94e6-256f4d51d570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.760953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dc402c74-8df3-4a66-94e6-256f4d51d570 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:47.762324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dc402c74-8df3-4a66-94e6-256f4d51d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:47.824977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 158/323] 10.222.0.71 () {60 vars in 1134 bytes} [Mon May 16 14:55:47 2022] POST /identity/v3/groups => generated 226 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:55:47.826505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a1aed-c17e-4c25-9888-ee641192dfe9 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.826795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a1aed-c17e-4c25-9888-ee641192dfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.827055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 166/324] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:47.827305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a1aed-c17e-4c25-9888-ee641192dfe9 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.828912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 159/325] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:47.834543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225150-b63d-4a8f-93c2-d9fb1a5d9e43 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:47.835895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225150-b63d-4a8f-93c2-d9fb1a5d9e43 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:47.835895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225150-b63d-4a8f-93c2-d9fb1a5d9e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:47.851458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a1aed-c17e-4c25-9888-ee641192dfe9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:47.857733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c5225150-b63d-4a8f-93c2-d9fb1a5d9e43 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:47.858958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5225150-b63d-4a8f-93c2-d9fb1a5d9e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:47.932710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 167/326] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:47.956262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225150-b63d-4a8f-93c2-d9fb1a5d9e43 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:47.962862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a1aed-c17e-4c25-9888-ee641192dfe9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.003341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.004187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.004473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.004804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.019031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:48.027704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:48.028778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:48.029569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 160/327] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:48.034613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:48.060257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225150-b63d-4a8f-93c2-d9fb1a5d9e43 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.096871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.097602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.097885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.098166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.109200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:48.117105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:48.118000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:48.118782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 168/328] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:48.124829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:48.169545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89905d-cb1c-4aa7-96b6-be9e1987f9bf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.206118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.206791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.207072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.207345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.208142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:48.208445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:48.209833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:48.222030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 161/329] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:48.227002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:48.233696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f9bcd-eea7-401b-bac4-26995b1f4602 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.271907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.272635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.272918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.273193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.273948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:48.274298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:48.276573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:48.288860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 169/330] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:48.297695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:48.350294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da05cd4-480e-4a2d-bfdd-9f5773c670ed None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.388707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.389704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.390107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.390485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] PATH_INFO: `/v3/users/a26140679fc045b8a093ff71e118772b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.391848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] RBAC: Authorizing `identity:get_user(user_id=a26140679fc045b8a093ff71e118772b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:48.393379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:48.395235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 162/331] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:48 2022] GET /identity/v3/users/a26140679fc045b8a093ff71e118772b => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:48.400819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:48.414178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe2b83-07aa-4705-8e2b-143a854a24f0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.456761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.458000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.458482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.458947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] PATH_INFO: `/v3/users/9dc57f8c9e8e4235a47e31fce2ea8701` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.460674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] RBAC: Authorizing `identity:get_user(user_id=9dc57f8c9e8e4235a47e31fce2ea8701)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:48.462363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:48.466108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 170/332] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:48 2022] GET /identity/v3/users/9dc57f8c9e8e4235a47e31fce2ea8701 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:48.472828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:48.514860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479211da-5e49-4bfe-a6b0-316fb2d4b86e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.551278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516b133-2f39-4c44-b645-ca5df1a67627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.552356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516b133-2f39-4c44-b645-ca5df1a67627 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.552842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516b133-2f39-4c44-b645-ca5df1a67627 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.553324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516b133-2f39-4c44-b645-ca5df1a67627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.554357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516b133-2f39-4c44-b645-ca5df1a67627 None admin] RBAC: Adding query filter params (role.id=70fe87cb18bc40b69190efb5dd0a933b, user.id=a26140679fc045b8a093ff71e118772b, scope.system=all) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:48.554885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516b133-2f39-4c44-b645-ca5df1a67627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:48.556631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516b133-2f39-4c44-b645-ca5df1a67627 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:48.570340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 163/333] 10.222.0.71 () {56 vars in 1308 bytes} [Mon May 16 14:55:48 2022] GET /identity/v3/role_assignments?user.id=a26140679fc045b8a093ff71e118772b&scope.system=all&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:48.587815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c0bb-320f-49b5-baf9-e651744a6337 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:48.643316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33886e1b-16bb-4368-9092-947e61cdc9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:48.644723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886e1b-16bb-4368-9092-947e61cdc9e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:48.645224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886e1b-16bb-4368-9092-947e61cdc9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:48.645705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886e1b-16bb-4368-9092-947e61cdc9e0 None admin] PATH_INFO: `/v3/system/users/9dc57f8c9e8e4235a47e31fce2ea8701/roles/1a29288217364c61b38b69e4e823a4d5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:48.647536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33886e1b-16bb-4368-9092-947e61cdc9e0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9dc57f8c9e8e4235a47e31fce2ea8701, role_id=1a29288217364c61b38b69e4e823a4d5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:48.649262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33886e1b-16bb-4368-9092-947e61cdc9e0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:48.731957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 171/334] 10.222.0.71 () {58 vars in 1334 bytes} [Mon May 16 14:55:48 2022] PUT /identity/v3/system/users/9dc57f8c9e8e4235a47e31fce2ea8701/roles/1a29288217364c61b38b69e4e823a4d5 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 14:55:50.669470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0906fe-f992-42ac-8d0c-1b1eb9f18e62 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:50.670573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0906fe-f992-42ac-8d0c-1b1eb9f18e62 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:50.671216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0906fe-f992-42ac-8d0c-1b1eb9f18e62 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:50.673283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 164/335] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:50.679236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041316c-a17b-402b-b337-18e1b1e5a619 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:50.680216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041316c-a17b-402b-b337-18e1b1e5a619 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:50.680702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041316c-a17b-402b-b337-18e1b1e5a619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:50.704310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3041316c-a17b-402b-b337-18e1b1e5a619 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:50.705858 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3041316c-a17b-402b-b337-18e1b1e5a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:50.790290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 172/336] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:50.816337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0906fe-f992-42ac-8d0c-1b1eb9f18e62 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:50.928144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0906fe-f992-42ac-8d0c-1b1eb9f18e62 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:50.966399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ced48-ec81-4923-9853-ab354860d37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:50.967209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ced48-ec81-4923-9853-ab354860d37f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:50.967548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ced48-ec81-4923-9853-ab354860d37f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:50.967854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ced48-ec81-4923-9853-ab354860d37f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:50.978289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ced48-ec81-4923-9853-ab354860d37f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-457ced48-ec81-4923-9853-ab354860d37f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:50.987801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:50.988738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:50.989545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:50.990177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a12e1-50a6-491f-84c2-232a14f4581d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:50.990177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a12e1-50a6-491f-84c2-232a14f4581d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:50.990177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a12e1-50a6-491f-84c2-232a14f4581d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:50.990717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 173/337] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:50.995727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 165/338] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:51.000873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a12e1-50a6-491f-84c2-232a14f4581d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.005081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06fd7c-35a9-4c51-b427-40b2e8da6467 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.005367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06fd7c-35a9-4c51-b427-40b2e8da6467 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.005643 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06fd7c-35a9-4c51-b427-40b2e8da6467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.032626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-dc06fd7c-35a9-4c51-b427-40b2e8da6467 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:51.033838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dc06fd7c-35a9-4c51-b427-40b2e8da6467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:51.124954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a12e1-50a6-491f-84c2-232a14f4581d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:51.165607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:51.166965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.167458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.168789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.169800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:51.170331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:51.172119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:51.172914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 166/339] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:51.174301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3efb5-6446-41e9-861f-dc45b7db1fdf None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.174583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3efb5-6446-41e9-861f-dc45b7db1fdf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.174855 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3efb5-6446-41e9-861f-dc45b7db1fdf None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.180029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 167/340] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:51.186342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba1a80-9101-44b8-b5be-78d2eb8690c0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.186624 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba1a80-9101-44b8-b5be-78d2eb8690c0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.186901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba1a80-9101-44b8-b5be-78d2eb8690c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.189309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 174/341] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:51.195155 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.216632 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-85ba1a80-9101-44b8-b5be-78d2eb8690c0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:51.217816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-85ba1a80-9101-44b8-b5be-78d2eb8690c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:51.325785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08521c-afd4-41dd-9ba8-5f5f1c00efc5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:51.365265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:51.366289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.366789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.367271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.382378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 168/342] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:51.383374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba1a80-9101-44b8-b5be-78d2eb8690c0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.479437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:51.481954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:51.522367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba1a80-9101-44b8-b5be-78d2eb8690c0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:51.557525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:51.558416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.559053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.559053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.569822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:51.578781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:51.579736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:51.580603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 169/343] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:51.581263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.594700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 175/344] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:55:51.599973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.686373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1b81e-4570-4b4a-aae2-4acef1a23bc5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:51.709156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3364e0-cc32-4bb9-b4d7-61968704ddac None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:51.723786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:51.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.725108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.725388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.738674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:51.749384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:51.750317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:51.751132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 170/345] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:51.753548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:51.754449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.754648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.755125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.755599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.756593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:51.757098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:51.758691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:51.772887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 176/346] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:51.774068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.872050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f31c75-aabe-49e3-a779-19f9ba421d44 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:51.924792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:51.925658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.925933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.926203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.927546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:51.929016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:51.930718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 171/347] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:51 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:51.931474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:51.933824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714d473-758f-4758-95fa-15cb39b5ceeb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:51.994174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:51.994776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:51.995064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:51.995342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:51.996371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:51.998961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.000397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.011611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 177/348] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:52.016879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:52.122708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8bd33-82ee-4547-92a5-f640bebe6abf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:52.188274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:52.188936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.189215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.189491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.204538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620f96-fdd8-4faa-a3bc-f243194902d1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:52.275302 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:52.276142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.276427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.276705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.293941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.295449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.297471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 178/349] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:52.298104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:52.325325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.326799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.435032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 172/350] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:52.436171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:52.444289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f70ea-6414-4751-a394-4fcd5263b072 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:52.484291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eae735-9eb5-4d3d-8d42-3a70d54885c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:52.485301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eae735-9eb5-4d3d-8d42-3a70d54885c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.485753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eae735-9eb5-4d3d-8d42-3a70d54885c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.486166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eae735-9eb5-4d3d-8d42-3a70d54885c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.545656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163cfee-758e-4181-8d0d-486a7f3fd655 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:52.555785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eae735-9eb5-4d3d-8d42-3a70d54885c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.557474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eae735-9eb5-4d3d-8d42-3a70d54885c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.582454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:52.583132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.583425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.583737 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.584593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=712891263b4f468fb3ce3dda3aeaed04) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:52.584961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.586525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.593970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 173/351] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:52.594834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:52.669862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 179/352] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:52.671541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56249c5-fc1b-4f24-9ceb-a3225be6c90e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.671995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56249c5-fc1b-4f24-9ceb-a3225be6c90e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.672449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56249c5-fc1b-4f24-9ceb-a3225be6c90e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.673972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 180/353] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:52.682079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe607d-a03c-42f1-8f23-f9beacbc5e74 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.682580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe607d-a03c-42f1-8f23-f9beacbc5e74 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.683012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe607d-a03c-42f1-8f23-f9beacbc5e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.700721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-63fe607d-a03c-42f1-8f23-f9beacbc5e74 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:52.702154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-63fe607d-a03c-42f1-8f23-f9beacbc5e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:52.711410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77abc09-9d24-422a-b180-0397ecee2b4a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:52.749130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:52.750382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.750382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.750596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.751763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.753406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.755420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 174/354] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:52.756255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:52.785105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 181/355] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:52.786679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe607d-a03c-42f1-8f23-f9beacbc5e74 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:52.894037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250aad21-a4ea-4a5b-a59b-05beec614d0b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:52.924440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe607d-a03c-42f1-8f23-f9beacbc5e74 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:52.934633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d383a-b12d-4717-9897-b7ede343a9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:52.935841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d383a-b12d-4717-9897-b7ede343a9db None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.936270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d383a-b12d-4717-9897-b7ede343a9db None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.936741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d383a-b12d-4717-9897-b7ede343a9db None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.939082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d383a-b12d-4717-9897-b7ede343a9db None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.940811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d383a-b12d-4717-9897-b7ede343a9db None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.943008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 175/356] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:52.944556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e675f-6487-437e-b4cb-5eb07ec9a9b6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.945043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e675f-6487-437e-b4cb-5eb07ec9a9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.945513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e675f-6487-437e-b4cb-5eb07ec9a9b6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.946944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 176/357] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:52.947867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e675f-6487-437e-b4cb-5eb07ec9a9b6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:52.969033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:52.970230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:52.970506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:52.970925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:52.972269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:52.973867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:52.975968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 182/358] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:52.976961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.076186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e675f-6487-437e-b4cb-5eb07ec9a9b6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.108000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4c13-45dd-4696-a275-15b122ea8471 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.117957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abc03f-0192-456f-ab8b-b094281115f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.119046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abc03f-0192-456f-ab8b-b094281115f6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.119566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abc03f-0192-456f-ab8b-b094281115f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.120035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abc03f-0192-456f-ab8b-b094281115f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.131260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abc03f-0192-456f-ab8b-b094281115f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-45abc03f-0192-456f-ab8b-b094281115f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:53.139819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:53.140755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 14:55:53.141556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:53.143673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 177/359] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:53.145289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306492ab-55d3-4eda-b0f0-99541ae70bc7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.145735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306492ab-55d3-4eda-b0f0-99541ae70bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.146157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306492ab-55d3-4eda-b0f0-99541ae70bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.152786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.154644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.154928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.155200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.156474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:53.157922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:53.159554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 183/360] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:52 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:53.160860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.169618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-306492ab-55d3-4eda-b0f0-99541ae70bc7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:53.171818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-306492ab-55d3-4eda-b0f0-99541ae70bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:53.256814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 178/361] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:53.258126 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306492ab-55d3-4eda-b0f0-99541ae70bc7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.294772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cdc7f-1ccc-4765-bf37-4f44a3689d1d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.342252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.342932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.343218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.343494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.344486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:53.344810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:53.346305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:53.353602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 184/362] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:53.354516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.389509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306492ab-55d3-4eda-b0f0-99541ae70bc7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.437394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.437831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.439440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.439759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.440557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:53.440866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:53.442324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:53.453428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 179/363] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:53.454300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.500716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3149d43-9c82-466d-8163-69f0797bffc5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.557022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79d8b6-8c5b-4609-aa30-c161aa9cedea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.557729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79d8b6-8c5b-4609-aa30-c161aa9cedea None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.558011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79d8b6-8c5b-4609-aa30-c161aa9cedea None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.558284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79d8b6-8c5b-4609-aa30-c161aa9cedea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.559709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79d8b6-8c5b-4609-aa30-c161aa9cedea None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=712891263b4f468fb3ce3dda3aeaed04) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:53.560029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79d8b6-8c5b-4609-aa30-c161aa9cedea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:53.561527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79d8b6-8c5b-4609-aa30-c161aa9cedea None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:53.569904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 185/364] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:53.570712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd065330-cd47-4cc2-8dca-15da3c6e706e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.571002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd065330-cd47-4cc2-8dca-15da3c6e706e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.571279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd065330-cd47-4cc2-8dca-15da3c6e706e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.572851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 186/365] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:53.573622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd065330-cd47-4cc2-8dca-15da3c6e706e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.591413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ab73-7ec0-459b-9333-699f659f3ef3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.630161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a82e7a-ace9-4e96-a423-bf065262c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.631355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a82e7a-ace9-4e96-a423-bf065262c55f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.631878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a82e7a-ace9-4e96-a423-bf065262c55f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.632420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a82e7a-ace9-4e96-a423-bf065262c55f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.645657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a82e7a-ace9-4e96-a423-bf065262c55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-77a82e7a-ace9-4e96-a423-bf065262c55f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:53.655320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:53.656314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:53.657120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:53.659714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 180/366] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:53.661337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855bd27-9d82-4b57-960d-1e9901b7e5b7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.661871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855bd27-9d82-4b57-960d-1e9901b7e5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.662356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855bd27-9d82-4b57-960d-1e9901b7e5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.681577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d855bd27-9d82-4b57-960d-1e9901b7e5b7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:53.683182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d855bd27-9d82-4b57-960d-1e9901b7e5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:53.708103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd065330-cd47-4cc2-8dca-15da3c6e706e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.749653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.750557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.750838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.751115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] PATH_INFO: `/v3/users/9dc57f8c9e8e4235a47e31fce2ea8701` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.752491 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] RBAC: Authorizing `identity:get_user(user_id=9dc57f8c9e8e4235a47e31fce2ea8701)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:53.754132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:53.756209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 187/367] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/users/9dc57f8c9e8e4235a47e31fce2ea8701 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:53.756990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.760310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 181/368] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:53.768759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855bd27-9d82-4b57-960d-1e9901b7e5b7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.887558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b9a58-2b40-4d2f-8401-1b7bb2aa66c0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.922169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855bd27-9d82-4b57-960d-1e9901b7e5b7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:53.929903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.931025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.931310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.931589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.932374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:53.933047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:53.935018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:53.946100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 188/369] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:53.947113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:53.967729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:53.968444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:53.968726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:53.968995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:53.969830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] RBAC: Adding query filter params (role.id=1a29288217364c61b38b69e4e823a4d5, user.id=9dc57f8c9e8e4235a47e31fce2ea8701, scope.system=all) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:53.970137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:53.972521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:53.987552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 182/370] 10.222.0.71 () {56 vars in 1308 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/role_assignments?user.id=9dc57f8c9e8e4235a47e31fce2ea8701&scope.system=all&role.id=1a29288217364c61b38b69e4e823a4d5 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:53.988435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:54.072616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f901375-a0bb-455e-912e-8f36a6c0cafe None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:54.126192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:54.127034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:54.127321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:54.127853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:54.141034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:54.158545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:54.159498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:54.160350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 189/371] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:54.167244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:54.270338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ee30-1411-4722-ae2f-2e272df95f79 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:54.322069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853004c-70e0-4abd-85aa-76ba713b4f5e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:54.351521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:54.352308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:54.352591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:54.353721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:54.390246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:54.390937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:54.391217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:54.391494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:54.392350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:54.392660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:54.394034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:54.409403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 190/372] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:54.413561 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:54.483791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:54.485253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:54.598482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62d6a8-aabe-4d57-91b4-55f2d27cb42a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:54.658429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:54.659113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:54.659397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:54.659766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:54.670693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 183/373] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:54.677138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:54.757166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:54.758383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:54.826734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d2b29-622e-47e4-8485-7852a2d40975 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:54.875818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:54.877237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:54.877708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:54.878171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:54.879370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 191/374] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:55:54.879786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:54.881359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:54.887971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 184/375] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:54 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:54.893517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:54.893715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:55.005647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d12f4-ae7a-4730-839b-e6c5020f2d4d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:55.049824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:55.050932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.051382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.051836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] PATH_INFO: `/v3/projects/63e6523cee6045dabb771a1e00b70243` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.053618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] RBAC: Authorizing `identity:get_project(project_id=63e6523cee6045dabb771a1e00b70243)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:55.055311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:55.057911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 185/376] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:54 2022] GET /identity/v3/projects/63e6523cee6045dabb771a1e00b70243 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:55.064306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:55.109808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1a405-cdf9-48aa-8d5d-c37ff662eeff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:55.167209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:55.168786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.169043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.169415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.170616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:55.172239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:55.173759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 192/377] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:54 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:55.175190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31892cab-0916-4934-ae79-dd9c7fa52d72 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:55.201659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:55.220261 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:55.221365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.221794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.222213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] PATH_INFO: `/v3/groups/52fa636992da4522ae4dc32581e68365` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.228679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] RBAC: Authorizing `identity:get_group(group_id=52fa636992da4522ae4dc32581e68365)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:55.231004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:55.233377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 186/378] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:55 2022] GET /identity/v3/groups/52fa636992da4522ae4dc32581e68365 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:55.239118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:55.343405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41d7c9-a9d2-47b6-a3cd-696b5c85ae9d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:55.377619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c9e69-6c77-4bda-8b34-4e635d575eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:55.378522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c9e69-6c77-4bda-8b34-4e635d575eac None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.378949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c9e69-6c77-4bda-8b34-4e635d575eac None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.379389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c9e69-6c77-4bda-8b34-4e635d575eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.380422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c9e69-6c77-4bda-8b34-4e635d575eac None admin] RBAC: Adding query filter params (group.id=52fa636992da4522ae4dc32581e68365, role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=63e6523cee6045dabb771a1e00b70243) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:55.380888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c9e69-6c77-4bda-8b34-4e635d575eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:55.382314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c9e69-6c77-4bda-8b34-4e635d575eac None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:55.388888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 187/379] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:55:55 2022] GET /identity/v3/role_assignments?group.id=52fa636992da4522ae4dc32581e68365&scope.project.id=63e6523cee6045dabb771a1e00b70243&role.id=1b3a0717a878438ea224dec0c7639471 => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:55.393635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b7f3-b23c-4426-83e1-55ccdb2d5b5b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:55.469970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0f32a-d239-47c0-b713-dd4aeb9218c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:55.470642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0f32a-d239-47c0-b713-dd4aeb9218c4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.470922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0f32a-d239-47c0-b713-dd4aeb9218c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.471195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0f32a-d239-47c0-b713-dd4aeb9218c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.472064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0f32a-d239-47c0-b713-dd4aeb9218c4 None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:55.472375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0f32a-d239-47c0-b713-dd4aeb9218c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:55.473894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0f32a-d239-47c0-b713-dd4aeb9218c4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:55.485270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 193/380] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:55:55 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 260 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:55.647734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301e0c-422e-4cf6-a8e3-230e7e2b8e4d None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.648053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301e0c-422e-4cf6-a8e3-230e7e2b8e4d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.648295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301e0c-422e-4cf6-a8e3-230e7e2b8e4d None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.649684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 188/381] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:55.657152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcc208-1739-4340-957c-0a8ba33e94d1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.657469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcc208-1739-4340-957c-0a8ba33e94d1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.657720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcc208-1739-4340-957c-0a8ba33e94d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.680163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3abcc208-1739-4340-957c-0a8ba33e94d1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:55.681374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3abcc208-1739-4340-957c-0a8ba33e94d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:55.758301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 194/382] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:55.784333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e301e0c-422e-4cf6-a8e3-230e7e2b8e4d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:55.899699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e301e0c-422e-4cf6-a8e3-230e7e2b8e4d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:55.935776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:55.936633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:55.936919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:55.937197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:55.948028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:55.956675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:55.957587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:55.959160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 189/383] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:55.963692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcc208-1739-4340-957c-0a8ba33e94d1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:56.075919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcc208-1739-4340-957c-0a8ba33e94d1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:56.116305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:56.116986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:56.117267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:56.117545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:56.118300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:56.118604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:56.120126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:56.135737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 195/384] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:56.141540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:56.253161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b94e-103f-49dc-b1d7-fe5af01afab5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:56.292634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:56.294150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:56.294432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:56.294704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:56.295978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:56.297435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:56.299376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 190/385] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:56 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:56.304644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:56.407148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aab25-a128-402f-a0fd-00927d9076f3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:56.446010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53855382-f40d-4b70-b3e3-12b30c9b8b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:56.446935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53855382-f40d-4b70-b3e3-12b30c9b8b66 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:56.447272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53855382-f40d-4b70-b3e3-12b30c9b8b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:56.447554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53855382-f40d-4b70-b3e3-12b30c9b8b66 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:56.448790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53855382-f40d-4b70-b3e3-12b30c9b8b66 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:56.450915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53855382-f40d-4b70-b3e3-12b30c9b8b66 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:56.452961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 196/386] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:56 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:56.458592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:56.578684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8529c77-7f37-4067-a388-33920b7f6898 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:56.619520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301ab07-b4b9-41b1-aa22-478354fdedb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:56.620432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301ab07-b4b9-41b1-aa22-478354fdedb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:56.620711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301ab07-b4b9-41b1-aa22-478354fdedb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:56.620968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301ab07-b4b9-41b1-aa22-478354fdedb3 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:56.623715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301ab07-b4b9-41b1-aa22-478354fdedb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=712891263b4f468fb3ce3dda3aeaed04, group_id=6b5904a50341475f9d4fafb5b39b9aac, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:56.625234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301ab07-b4b9-41b1-aa22-478354fdedb3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:56.737699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 191/387] 10.222.0.71 () {58 vars in 1442 bytes} [Mon May 16 14:55:56 2022] PUT /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:56.800721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38bef9-ba73-4569-9f43-a58ca70ed9ec None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:56.800721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38bef9-ba73-4569-9f43-a58ca70ed9ec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:56.800721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38bef9-ba73-4569-9f43-a58ca70ed9ec None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:56.801499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 197/388] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:56.810706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cefcc6-7379-4152-bb8a-8555ce26f3cb None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:56.811361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cefcc6-7379-4152-bb8a-8555ce26f3cb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:56.811880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cefcc6-7379-4152-bb8a-8555ce26f3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:56.830013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-63cefcc6-7379-4152-bb8a-8555ce26f3cb None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:56.831604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-63cefcc6-7379-4152-bb8a-8555ce26f3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:56.903482 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 192/389] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:56.927362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38bef9-ba73-4569-9f43-a58ca70ed9ec None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.041262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38bef9-ba73-4569-9f43-a58ca70ed9ec None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.051278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f2b1-9b23-4ecb-b4d0-a50824c23217 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.051564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f2b1-9b23-4ecb-b4d0-a50824c23217 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.051814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f2b1-9b23-4ecb-b4d0-a50824c23217 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.054013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 193/390] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:57.068928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555e438-58e2-492d-b31e-0a74ef55153a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.069244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555e438-58e2-492d-b31e-0a74ef55153a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.069495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555e438-58e2-492d-b31e-0a74ef55153a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.091702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4555e438-58e2-492d-b31e-0a74ef55153a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:57.093224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4555e438-58e2-492d-b31e-0a74ef55153a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:57.099285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.099285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.099285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.099285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.123062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:57.131853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:57.132751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:57.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:57.136277 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 198/391] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:55:57.142480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.179833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 194/392] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:57.208818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555e438-58e2-492d-b31e-0a74ef55153a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.259618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65998-8ae7-4f01-8234-95030d1a08d2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.298062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.298645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.298994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.299192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.299989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:57.300296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:57.302148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:57.313289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 199/393] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:57.319421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.419768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555e438-58e2-492d-b31e-0a74ef55153a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.436934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1aa1cc-4509-43eb-8ad0-e9ca9e8a04fa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.475248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.476378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.476865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.477242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.478713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:57.480571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:57.487002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 200/394] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:57 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:57.488509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.495905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.497097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.497581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.498022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.521254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:57.531701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:57.532612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:55:57.533407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:57.537740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 195/395] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:57.543139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.597537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705359-6470-47d0-9e66-9a4a0ac6e947 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.631659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.632469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.632754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.633032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.634230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:57.639997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:57.639997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 201/396] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:57 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:57.643624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.723373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9081e67-891e-4b33-a44f-85c0f0ff0f8a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.748490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b066ca7-48b8-4a3d-919b-9e30baa4950e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.782423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.783314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.783599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.783902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.786139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] RBAC: Authorizing `identity:create_grant(project_id=712891263b4f468fb3ce3dda3aeaed04, group_id=6b5904a50341475f9d4fafb5b39b9aac, role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:57.787584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:57.798883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.799648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.799938 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.800218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.801031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:57.801442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:57.802964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:57.813097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 196/397] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:55:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:57.821549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:57.885242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 202/398] 10.222.0.71 () {58 vars in 1442 bytes} [Mon May 16 14:55:57 2022] PUT /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 14:55:57.947238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74ac3a-dce4-453f-8dbc-6f735dec6593 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:57.989721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:57.990655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:57.991030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:57.991313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:57.995897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:57.998307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:58.000709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 197/399] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:57 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:58.006132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:58.130342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43170-16a2-4959-b3d7-6b56320fe958 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:58.166754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c3bd-80e2-4fd0-9bf5-c7e076d6fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:58.167657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c3bd-80e2-4fd0-9bf5-c7e076d6fb04 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.167934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c3bd-80e2-4fd0-9bf5-c7e076d6fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.168924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c3bd-80e2-4fd0-9bf5-c7e076d6fb04 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.170248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c3bd-80e2-4fd0-9bf5-c7e076d6fb04 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:58.171723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c3bd-80e2-4fd0-9bf5-c7e076d6fb04 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:58.173873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 203/400] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:58 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:58.179530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:58.290955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebe4d-fd22-422f-bb28-fae1968372f0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:58.331125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a13ba0-ed43-4a19-823b-f3ae9d8110a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:58.332110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a13ba0-ed43-4a19-823b-f3ae9d8110a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.332397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a13ba0-ed43-4a19-823b-f3ae9d8110a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.332675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a13ba0-ed43-4a19-823b-f3ae9d8110a5 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.334830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a13ba0-ed43-4a19-823b-f3ae9d8110a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad1577623e54cb486ce87b0fb9cef96, group_id=6b5904a50341475f9d4fafb5b39b9aac, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:58.336484 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a13ba0-ed43-4a19-823b-f3ae9d8110a5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:58.447745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08d3c-2408-4700-8ded-eb27e70519f6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.448034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08d3c-2408-4700-8ded-eb27e70519f6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.448203 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08d3c-2408-4700-8ded-eb27e70519f6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.450064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 204/401] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:58.450871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 198/402] 10.222.0.71 () {58 vars in 1442 bytes} [Mon May 16 14:55:58 2022] PUT /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:55:58.455443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33eca2-3007-492e-bbd4-a35f25eb04c7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.455792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33eca2-3007-492e-bbd4-a35f25eb04c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.456164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33eca2-3007-492e-bbd4-a35f25eb04c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.481499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0a33eca2-3007-492e-bbd4-a35f25eb04c7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:58.482922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0a33eca2-3007-492e-bbd4-a35f25eb04c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:58.526253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb574b-1bfc-4d8c-9491-8d2f1331fe93 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.526253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb574b-1bfc-4d8c-9491-8d2f1331fe93 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.526253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb574b-1bfc-4d8c-9491-8d2f1331fe93 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.526830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 199/403] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:55:58.532598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abb8d-d786-478f-ae5b-94451d4fb92e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.539274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abb8d-d786-478f-ae5b-94451d4fb92e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.539274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abb8d-d786-478f-ae5b-94451d4fb92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.559733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-369abb8d-d786-478f-ae5b-94451d4fb92e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:58.562152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-369abb8d-d786-478f-ae5b-94451d4fb92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:58.599449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 205/404] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:58.625918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33eca2-3007-492e-bbd4-a35f25eb04c7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:58.715946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 200/405] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:55:58.739051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369abb8d-d786-478f-ae5b-94451d4fb92e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:58.798010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33eca2-3007-492e-bbd4-a35f25eb04c7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:58.837147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:58.837966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.838249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.838527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.851536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:58.854931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369abb8d-d786-478f-ae5b-94451d4fb92e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:58.868042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:58.868974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:55:58.869758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 206/406] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:55:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:58.878878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:58.900859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:58.901719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:58.902002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:58.902328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:58.923225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:55:58.934370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:55:58.936684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:55:58.938648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 201/407] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:55:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:55:58.947332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:59.056622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ea3e-912b-4349-88e9-4c0fce793933 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.111862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc2967-e704-4127-9173-17764564c8cb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.145663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.146203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.146523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.146806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.147079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.147756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.147873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:59.148148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.148552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.149010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.149515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.150155 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:55:59.150646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.152239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.158007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 202/408] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:59.166179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 207/409] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:55:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:59.167098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:59.173020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:59.287240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b4a40-b890-4073-b9f2-60c0223ae2e0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.330264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.330264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.330264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.330264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] PATH_INFO: `/v3/groups/52fa636992da4522ae4dc32581e68365` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.330631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] RBAC: Authorizing `identity:get_group(group_id=52fa636992da4522ae4dc32581e68365)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.330631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.330631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 203/410] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:59 2022] GET /identity/v3/groups/52fa636992da4522ae4dc32581e68365 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:59.333842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:59.399205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179dee2-692c-49ce-93d7-33b9e26f84ed None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.435861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc34f-0197-4c1a-8f93-48cb2237ddbf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.470030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.470898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.471178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.471481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] PATH_INFO: `/v3/projects/63e6523cee6045dabb771a1e00b70243` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.472700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] RBAC: Authorizing `identity:get_project(project_id=63e6523cee6045dabb771a1e00b70243)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.474169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.476213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 204/411] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:59 2022] GET /identity/v3/projects/63e6523cee6045dabb771a1e00b70243 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:55:59.482414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:59.483346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.484494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.484986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.485460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.486907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.488563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.490692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 208/412] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:55:59 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:59.495683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:59.630711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37664228-c1d9-490c-9dd8-dd92d46c8373 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.681372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34e6a9-4a6e-4c67-9cbf-63500476cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.682614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e6a9-4a6e-4c67-9cbf-63500476cd00 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.683112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e6a9-4a6e-4c67-9cbf-63500476cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.683583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e6a9-4a6e-4c67-9cbf-63500476cd00 None admin] PATH_INFO: `/v3/projects/63e6523cee6045dabb771a1e00b70243/groups/52fa636992da4522ae4dc32581e68365/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.690457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34e6a9-4a6e-4c67-9cbf-63500476cd00 None admin] RBAC: Authorizing `identity:create_grant(project_id=63e6523cee6045dabb771a1e00b70243, group_id=52fa636992da4522ae4dc32581e68365, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.692188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34e6a9-4a6e-4c67-9cbf-63500476cd00 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.708862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a545e1-882d-4e9b-9cbb-7661f9028ff4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.743260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.744422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.744873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.745312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.747382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.749018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.751229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 209/413] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:55:59 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:55:59.756413 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:55:59.767213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 205/414] 10.222.0.71 () {58 vars in 1442 bytes} [Mon May 16 14:55:59 2022] PUT /identity/v3/projects/63e6523cee6045dabb771a1e00b70243/groups/52fa636992da4522ae4dc32581e68365/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 14:55:59.806158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6086ee1-65d9-48f3-b27d-4ba7015ceadf None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.806672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6086ee1-65d9-48f3-b27d-4ba7015ceadf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.807155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6086ee1-65d9-48f3-b27d-4ba7015ceadf None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.808750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 206/415] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:55:59.815116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb62452-3c09-4a22-8fa4-43f1f7aeedf1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.815726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb62452-3c09-4a22-8fa4-43f1f7aeedf1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.816121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb62452-3c09-4a22-8fa4-43f1f7aeedf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.833488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6fb62452-3c09-4a22-8fa4-43f1f7aeedf1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:55:59.834860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6fb62452-3c09-4a22-8fa4-43f1f7aeedf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:55:59.883751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b24cb-8285-4c86-834f-2d8aa78014ee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:55:59.937188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:55:59.938465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:55:59.938946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:55:59.939416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:55:59.943306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 207/416] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:55:59.944320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad1577623e54cb486ce87b0fb9cef96, group_id=6b5904a50341475f9d4fafb5b39b9aac, role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:55:59.946036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:55:59.967746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb62452-3c09-4a22-8fa4-43f1f7aeedf1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:00.024166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 210/417] 10.222.0.71 () {58 vars in 1442 bytes} [Mon May 16 14:55:59 2022] PUT /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96/groups/6b5904a50341475f9d4fafb5b39b9aac/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 14:56:00.080862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb62452-3c09-4a22-8fa4-43f1f7aeedf1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:00.119664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:00.120537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:00.120819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:00.121090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:00.133388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:00.142314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:00.143227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:00.144068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 208/418] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:55:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:00.149803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:00.270031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e35729-6513-4e67-a386-c0b36e9cacdc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:00.305241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:00.305911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:00.306190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:00.306462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:00.316598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:00.316598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:00.316598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:00.318768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 211/419] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:56:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:00.324226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:00.426086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360ec06-9e47-415b-aec4-e83f0a239746 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:00.459885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:00.460538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:00.460821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:00.461090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:00.529438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:00.530890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:00.633757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 209/420] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:00.639986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:00.755607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5887598-8e6a-4453-9a61-c97dba604869 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:00.799828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6069c-5fb1-4b08-afb2-690150be71bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:00.800692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6069c-5fb1-4b08-afb2-690150be71bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:00.800969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6069c-5fb1-4b08-afb2-690150be71bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:00.801237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6069c-5fb1-4b08-afb2-690150be71bd None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:00.802417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6069c-5fb1-4b08-afb2-690150be71bd None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:00.803871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6069c-5fb1-4b08-afb2-690150be71bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:00.805555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 212/421] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:56:00 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:00.811746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:00.816938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2871d1-0d58-4f19-b1a9-d648020ff514 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:00.817287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2871d1-0d58-4f19-b1a9-d648020ff514 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:00.817562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2871d1-0d58-4f19-b1a9-d648020ff514 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:00.818937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 213/422] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:00.824698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39090427-f45d-4b2f-8249-c94e494fb700 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:00.825584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39090427-f45d-4b2f-8249-c94e494fb700 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:00.825584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39090427-f45d-4b2f-8249-c94e494fb700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:00.843972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-39090427-f45d-4b2f-8249-c94e494fb700 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:00.845728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-39090427-f45d-4b2f-8249-c94e494fb700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:00.969165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 214/423] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:00.995927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6bfd2-63eb-4f17-ae98-c2f0563ec7a9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.014725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39090427-f45d-4b2f-8249-c94e494fb700 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.048055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.048922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.049204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.049477 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.053097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:01.057891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:01.060474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 210/424] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:56:00 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:01.065943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.195941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3113d7-d199-4621-ab4b-1d3ead851e47 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.207978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39090427-f45d-4b2f-8249-c94e494fb700 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.240611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.241305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.241678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.241965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.243037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=712891263b4f468fb3ce3dda3aeaed04) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:01.243429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:01.245083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:01.245630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64f11b-16c8-4e33-84e1-2e63e4ee7eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.247496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64f11b-16c8-4e33-84e1-2e63e4ee7eec None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.248054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64f11b-16c8-4e33-84e1-2e63e4ee7eec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.248526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64f11b-16c8-4e33-84e1-2e63e4ee7eec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.253727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 211/425] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:01.274946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64f11b-16c8-4e33-84e1-2e63e4ee7eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3e64f11b-16c8-4e33-84e1-2e63e4ee7eec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:01.283866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:01.284865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:56:01.285654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:01.288484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 215/426] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:01.295668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.463488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c25d0-117d-4f14-b09f-2419423a7579 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.468822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c334e8-a26e-41a6-ac74-61e0edaf8d3a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.469105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c334e8-a26e-41a6-ac74-61e0edaf8d3a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.469382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c334e8-a26e-41a6-ac74-61e0edaf8d3a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.471049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 216/427] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:01.476053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61387-aa3e-4072-981a-675fec91534a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.476336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61387-aa3e-4072-981a-675fec91534a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.476614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61387-aa3e-4072-981a-675fec91534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.499917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-aeb61387-aa3e-4072-981a-675fec91534a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:01.501152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aeb61387-aa3e-4072-981a-675fec91534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:01.510644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.511314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.511593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.511909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.512695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:01.513001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:01.514373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:01.522773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 212/428] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:01.528271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.575844 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 217/429] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:01.606963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61387-aa3e-4072-981a-675fec91534a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.636803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc22b-f427-446b-9f19-c0b846e250b4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.671726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.672731 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.673209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.673678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.719216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61387-aa3e-4072-981a-675fec91534a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.743748 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:01.745525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:01.756822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.757631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.758167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.758167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.773876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:01.782294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:01.783199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 14:56:01.784017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:01.787756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 218/430] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:01.789354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.856720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 213/431] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:56:01.858200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.911549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afdbad-88b4-4380-beac-3b0963421bd5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.946850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.947525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.948426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.948705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.950110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:01.950629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:01.952054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:01.958249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399d94c-c339-47eb-a58d-0f0a1f5f5ae1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:01.967267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 219/432] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:01.968793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:01.995515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:01.996833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:01.997763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:01.998250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] PATH_INFO: `/v3/projects/712891263b4f468fb3ce3dda3aeaed04` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:01.999663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] RBAC: Authorizing `identity:get_project(project_id=712891263b4f468fb3ce3dda3aeaed04)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:02.001329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:02.003502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 214/433] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/projects/712891263b4f468fb3ce3dda3aeaed04 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:02.008972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:02.074575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77389daa-2598-40c6-b777-3a0bb6eacb26 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:02.109500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:02.110275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.110559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.110829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.111523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb9f8-e300-433e-9a99-b1131e599103 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:02.145578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:02.146812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.147289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.147803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.149187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:02.150848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:02.153053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 215/434] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:56:02 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:02.158764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:02.182051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:02.183668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:02.259657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5b557-d67b-4ce4-843b-e47216e84190 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:02.290902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 220/435] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:02.292441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:02.295937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:02.296836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.297348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.297819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.298840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=712891263b4f468fb3ce3dda3aeaed04) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:02.299346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:02.301576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:02.308362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 216/436] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:56:02 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=712891263b4f468fb3ce3dda3aeaed04&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:02.412382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9e75-5ce3-437d-9ece-5d17e052ce45 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:02.454427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:02.455317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.455594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.455905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.457111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:02.458558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:02.460673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 221/437] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:56:02 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:02.466980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:02.601923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff0b0d-bc85-49ec-9d6e-8e43040bec08 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:02.635678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27173dc8-417c-4ef7-8b1c-b7660f51fb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:02.636547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27173dc8-417c-4ef7-8b1c-b7660f51fb64 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.636829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27173dc8-417c-4ef7-8b1c-b7660f51fb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.637170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27173dc8-417c-4ef7-8b1c-b7660f51fb64 None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.638962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27173dc8-417c-4ef7-8b1c-b7660f51fb64 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:02.640437 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27173dc8-417c-4ef7-8b1c-b7660f51fb64 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:02.642371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 217/438] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:56:02 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:02.647924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:02.734542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710bd8e-94f0-4451-950a-d51d2939d9f6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.734895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710bd8e-94f0-4451-950a-d51d2939d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.735100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710bd8e-94f0-4451-950a-d51d2939d9f6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.736627 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 218/439] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:02.741661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b796-e6ba-40c7-833c-c1ea338e7cd0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.741944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b796-e6ba-40c7-833c-c1ea338e7cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.742223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b796-e6ba-40c7-833c-c1ea338e7cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.761835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1d03b796-e6ba-40c7-833c-c1ea338e7cd0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:02.763923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1d03b796-e6ba-40c7-833c-c1ea338e7cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:02.766340 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b420e-1fb8-4264-b9ed-ea6f97e7c455 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:02.803005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:02.803694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.803984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.804256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.805271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=70fe87cb18bc40b69190efb5dd0a933b, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:02.805579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:02.807023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:02.815470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 222/440] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:56:02 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=70fe87cb18bc40b69190efb5dd0a933b => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:02.846764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 219/441] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:02.873283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:02.875951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d07d-335a-418b-be2f-d6a2a0bf0429 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.876238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d07d-335a-418b-be2f-d6a2a0bf0429 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.876510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d07d-335a-418b-be2f-d6a2a0bf0429 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.877895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 220/442] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:02.883742 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad43bb-3bdb-4a35-b66a-a1890feab5dc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:02.884028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad43bb-3bdb-4a35-b66a-a1890feab5dc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:02.884307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad43bb-3bdb-4a35-b66a-a1890feab5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:02.901923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8bad43bb-3bdb-4a35-b66a-a1890feab5dc None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:02.904134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8bad43bb-3bdb-4a35-b66a-a1890feab5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:02.973886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 221/443] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:02.982109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d07cc-8d6d-43a5-a68a-f4e3d9ac4a1e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:02.997366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad43bb-3bdb-4a35-b66a-a1890feab5dc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.020550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.021350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.021635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.021905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.034165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:03.042130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:03.043226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:03.044638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:03.046572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 223/444] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:03.052368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.111802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad43bb-3bdb-4a35-b66a-a1890feab5dc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.149337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.150159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.150443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.150719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.162637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:03.171664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:03.172841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:56:03.173925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:03.174574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7222b-8621-4253-9d33-d2b709a440a9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.174647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 222/445] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:03.177987 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.210835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.211503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.211853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.212125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.212861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:03.213392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.215040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.224199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 224/446] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:03.229331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.280796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff225-c0ef-4036-9698-dd093b4000d2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.312962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.313633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.313917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.314196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.314935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:03.317931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.317931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.322617 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e27c58-3508-47c7-b46e-c5f9afc51946 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.324262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 223/447] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:03.329646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.355146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.356688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.356973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.357245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.419582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.421068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.422719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424ad60-da83-4f17-ac38-a007e49753ac None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.453842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.454478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.454757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.455027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.512780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 225/448] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:56:03.517736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.518253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.519169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.610186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 224/449] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 14:56:03.610388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccbee6-34db-4467-bc39-b3a0932f3ab8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.615478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.642672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.643509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.643831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.644105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] PATH_INFO: `/v3/projects/63e6523cee6045dabb771a1e00b70243` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.645275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] RBAC: Authorizing `identity:get_project(project_id=63e6523cee6045dabb771a1e00b70243)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.646727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.648436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 226/450] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/projects/63e6523cee6045dabb771a1e00b70243 => generated 344 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:03.654380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.706525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1986970-f0a1-4eab-a30f-0d9aa2b098f4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.739002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.739878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.740158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.740427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] PATH_INFO: `/v3/projects/4ad1577623e54cb486ce87b0fb9cef96` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.741580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] RBAC: Authorizing `identity:get_project(project_id=4ad1577623e54cb486ce87b0fb9cef96)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.743000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.744697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 225/451] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/projects/4ad1577623e54cb486ce87b0fb9cef96 => generated 302 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:03.747753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6b5b-835b-498c-bb8e-c6506ec4627a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.750043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.780895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.781740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.782024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.782300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] PATH_INFO: `/v3/groups/52fa636992da4522ae4dc32581e68365` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.783471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] RBAC: Authorizing `identity:get_group(group_id=52fa636992da4522ae4dc32581e68365)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.784936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.786549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 227/452] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/groups/52fa636992da4522ae4dc32581e68365 => generated 229 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:03.792236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.842982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2371e4-e5d1-4a1f-8c01-c832dc7c3c1a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.875221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.876040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.876325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.876607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] PATH_INFO: `/v3/groups/6b5904a50341475f9d4fafb5b39b9aac` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.877781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] RBAC: Authorizing `identity:get_group(group_id=6b5904a50341475f9d4fafb5b39b9aac)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.879227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.880877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 226/453] 10.222.0.71 () {56 vars in 1180 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/groups/6b5904a50341475f9d4fafb5b39b9aac => generated 226 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:03.886360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:03.888201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9f4ad-7e93-4158-beee-f7c6be05c33c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:03.921274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27c82c-b1b7-449a-b129-7b6fedf4af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:03.921930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c82c-b1b7-449a-b129-7b6fedf4af73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:03.922216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c82c-b1b7-449a-b129-7b6fedf4af73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:03.922490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c82c-b1b7-449a-b129-7b6fedf4af73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:03.923313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c82c-b1b7-449a-b129-7b6fedf4af73 None admin] RBAC: Adding query filter params (group.id=52fa636992da4522ae4dc32581e68365, role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=63e6523cee6045dabb771a1e00b70243) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:03.923623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c82c-b1b7-449a-b129-7b6fedf4af73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:03.925080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c82c-b1b7-449a-b129-7b6fedf4af73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:03.930756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 228/454] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/role_assignments?group.id=52fa636992da4522ae4dc32581e68365&scope.project.id=63e6523cee6045dabb771a1e00b70243&role.id=1b3a0717a878438ea224dec0c7639471 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:03.983889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265e9a-61b3-47e5-9999-f87a5994555a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:04.018598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ef56a-beef-4a6a-b511-fe81a78cc8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:04.019250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ef56a-beef-4a6a-b511-fe81a78cc8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:04.019517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ef56a-beef-4a6a-b511-fe81a78cc8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:04.019819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ef56a-beef-4a6a-b511-fe81a78cc8b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:04.020658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ef56a-beef-4a6a-b511-fe81a78cc8b3 None admin] RBAC: Adding query filter params (group.id=6b5904a50341475f9d4fafb5b39b9aac, role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=4ad1577623e54cb486ce87b0fb9cef96) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:04.020969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ef56a-beef-4a6a-b511-fe81a78cc8b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:04.022424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ef56a-beef-4a6a-b511-fe81a78cc8b3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:04.029078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 227/455] 10.222.0.71 () {56 vars in 1376 bytes} [Mon May 16 14:56:03 2022] GET /identity/v3/role_assignments?group.id=6b5904a50341475f9d4fafb5b39b9aac&scope.project.id=4ad1577623e54cb486ce87b0fb9cef96&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:07.063964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d163-7751-4ecd-b8b2-3b85c0b152cf None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.064992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d163-7751-4ecd-b8b2-3b85c0b152cf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.065474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d163-7751-4ecd-b8b2-3b85c0b152cf None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.067276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 229/456] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:07.072939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07deab7-a81c-4824-be3e-01d70e3ad536 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.073765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07deab7-a81c-4824-be3e-01d70e3ad536 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.074235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07deab7-a81c-4824-be3e-01d70e3ad536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.082798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc139ce2-6c48-4ff2-86c7-c4ab0132355f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.083345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc139ce2-6c48-4ff2-86c7-c4ab0132355f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.083877 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc139ce2-6c48-4ff2-86c7-c4ab0132355f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.085683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 230/457] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:07.091380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070c1c-d5bf-4b1c-806a-0374b934d247 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.091586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070c1c-d5bf-4b1c-806a-0374b934d247 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.091847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070c1c-d5bf-4b1c-806a-0374b934d247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.095446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b07deab7-a81c-4824-be3e-01d70e3ad536 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:07.096614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b07deab7-a81c-4824-be3e-01d70e3ad536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:07.120253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f3070c1c-d5bf-4b1c-806a-0374b934d247 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:07.121409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f3070c1c-d5bf-4b1c-806a-0374b934d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:07.229387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 231/458] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:07.231842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 228/459] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:07.253908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5dad-e71f-4d18-bf98-665879207394 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.254406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5dad-e71f-4d18-bf98-665879207394 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.254823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5dad-e71f-4d18-bf98-665879207394 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.256355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 232/460] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:07.265486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94e39e-bccf-40af-9a6a-8e4b747bbb8c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.265976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94e39e-bccf-40af-9a6a-8e4b747bbb8c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.266388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94e39e-bccf-40af-9a6a-8e4b747bbb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.270393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea5dad-e71f-4d18-bf98-665879207394 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.287969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ac94e39e-bccf-40af-9a6a-8e4b747bbb8c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:07.289484 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ac94e39e-bccf-40af-9a6a-8e4b747bbb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:07.372237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 229/461] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:07.372861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94e39e-bccf-40af-9a6a-8e4b747bbb8c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.391337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea5dad-e71f-4d18-bf98-665879207394 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:07.427821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875837be-6221-4ac7-85c7-49d23fcdcfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:07.428951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875837be-6221-4ac7-85c7-49d23fcdcfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.429454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875837be-6221-4ac7-85c7-49d23fcdcfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.429919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875837be-6221-4ac7-85c7-49d23fcdcfa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.440367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875837be-6221-4ac7-85c7-49d23fcdcfa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:07.442095 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875837be-6221-4ac7-85c7-49d23fcdcfa1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-875837be-6221-4ac7-85c7-49d23fcdcfa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:07.460090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:07.461172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:07.462224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:07.462224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:07.462224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:07.462224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:07.463824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 233/462] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:07.465500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2548d57-04c7-4a7d-be16-03198d1c64de None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.466567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2548d57-04c7-4a7d-be16-03198d1c64de None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.466996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2548d57-04c7-4a7d-be16-03198d1c64de None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.468400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 234/463] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:07.469224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2548d57-04c7-4a7d-be16-03198d1c64de None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.477846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94e39e-bccf-40af-9a6a-8e4b747bbb8c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:07.510164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:07.510951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.511234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.511509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.522348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:07.523818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:07.542137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:07.542982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:07.543873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:07.543873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:07.543873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:07.543873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:07.543873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 230/464] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:07.544766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.572241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2548d57-04c7-4a7d-be16-03198d1c64de None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:07.610038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b127f-66fb-41d2-9cf5-cf89ad77747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:07.611045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b127f-66fb-41d2-9cf5-cf89ad77747d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.611319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b127f-66fb-41d2-9cf5-cf89ad77747d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.611593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b127f-66fb-41d2-9cf5-cf89ad77747d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.621062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b127f-66fb-41d2-9cf5-cf89ad77747d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:07.623568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b127f-66fb-41d2-9cf5-cf89ad77747d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e77b127f-66fb-41d2-9cf5-cf89ad77747d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:07.641012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:07.641879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:07.642711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:07.642711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:07.642711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:07.642711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:07.642711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 235/465] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:07.643371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d8cd2-2ffb-444c-b905-de353f8fdeb5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.643660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d8cd2-2ffb-444c-b905-de353f8fdeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.643933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d8cd2-2ffb-444c-b905-de353f8fdeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.649053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3148-26b1-4cc1-aaa4-0e1a35b33fe4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:07.660866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d32d8cd2-2ffb-444c-b905-de353f8fdeb5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:07.662043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d32d8cd2-2ffb-444c-b905-de353f8fdeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:07.683257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:07.683953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.684238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.684605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.685280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:07.685586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:07.686942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:07.698605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 231/466] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:07.699570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.737516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 236/467] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:07.738580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d8cd2-2ffb-444c-b905-de353f8fdeb5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.812242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62759b2-e508-4f8a-ae47-280dedbe49f0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:07.837473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d8cd2-2ffb-444c-b905-de353f8fdeb5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:07.845138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:07.845800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.846400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.846400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.847046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:07.847400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:07.848836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:07.860437 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 232/468] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:07.861407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.884883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:07.885520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.885802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.886076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.886800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:07.887108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:07.888727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:07.905072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 237/469] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:07.905072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:07.961972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801f5ad-4c3b-4fe7-a659-4097de2d7737 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:07.995017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:07.995701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:07.996000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:07.996257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:07.997027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:07.999132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:08.002784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9de9-e169-487a-9311-37b22d518dd9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:08.036369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:08.037161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.037441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.037712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.048234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:08.049683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:08.066086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:08.067410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:08.068837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:08.068837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:08.068837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:08.068837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:08.068837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 238/470] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:08.068837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:08.118656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 233/471] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:56:07 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:08.120361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:08.177429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414778d5-5f77-4f4c-a5f3-f468ec144c66 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:08.226828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:08.227536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.227862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.228145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.228891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:08.230423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:08.243253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80aedf-46ae-4159-8fb6-fe48a2be790e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:08.283495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b273d9-90ce-4adf-b680-7071f0b68fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:08.284212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b273d9-90ce-4adf-b680-7071f0b68fd8 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.284498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b273d9-90ce-4adf-b680-7071f0b68fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.284776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b273d9-90ce-4adf-b680-7071f0b68fd8 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.285498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b273d9-90ce-4adf-b680-7071f0b68fd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:08.286954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b273d9-90ce-4adf-b680-7071f0b68fd8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:08.310490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 239/472] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:56:08 2022] POST /identity/v3/users => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:08.310490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:08.396015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 234/473] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:56:08 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:08.436336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fd2eb-9cdf-4518-8f57-9afc3ebde76a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.436615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fd2eb-9cdf-4518-8f57-9afc3ebde76a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.436814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fd2eb-9cdf-4518-8f57-9afc3ebde76a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.438258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 235/474] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:08.442180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a1db-6b80-49ff-b0c0-2b3c107b1341 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:08.445890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eb302-e530-40bb-857c-ede8c338d299 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.446167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eb302-e530-40bb-857c-ede8c338d299 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.446493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eb302-e530-40bb-857c-ede8c338d299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.469924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2c2eb302-e530-40bb-857c-ede8c338d299 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:08.471236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2c2eb302-e530-40bb-857c-ede8c338d299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:08.495483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:08.496695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.497239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.498058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.499086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:08.499663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:08.501369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:08.516330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 240/475] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:08.521656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:08.559158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 236/476] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:08.590401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eb302-e530-40bb-857c-ede8c338d299 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:08.651932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a74911-7d6e-43a9-a0ad-012b12842b94 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:08.706577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:08.707427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.707897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.708287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.709124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:08.710662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:08.729071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eb302-e530-40bb-857c-ede8c338d299 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:08.765117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:08.765801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:08.766079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:08.766350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:08.767115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:08.768556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:08.789840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 241/477] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:56:08 2022] POST /identity/v3/users => generated 249 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:08.843964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.sql.core [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 16 14:56:08.843964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 16 14:56:08.843964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [parameters: {'id': 'ee1c3032bfb24da5801fd878649b45aa', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 16 14:56:08.843964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80232) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 16 14:56:08.865833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return ref.to_dict() May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 14:56:08.866756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application yield resource May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.rollback() May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.force_reraise() May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise self.value May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application session.commit() May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 16 14:56:08.867746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._prepare_impl() May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.flush() May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._flush(objects) May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application compat.raise_( May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application flush_context.execute() May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rec.execute(self) May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 16 14:56:08.868601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application _emit_insert_statements( May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application c = connection._execute_20( May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ret = self._execute_context( May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 14:56:08.869511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 14:56:08.870328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [parameters: {'id': 'ee1c3032bfb24da5801fd878649b45aa', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:08.873304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 16 14:56:08.875410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:08.879107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 237/478] 10.222.0.71 () {60 vars in 1131 bytes} [Mon May 16 14:56:08 2022] POST /identity/v3/roles => generated 144 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:08.885201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:08.995011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae3c1-8382-4f67-b5d9-b968fd6015aa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:09.035953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da93a7e-6598-4b88-a87d-1956e8831c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:09.037235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da93a7e-6598-4b88-a87d-1956e8831c9f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:09.037773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da93a7e-6598-4b88-a87d-1956e8831c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:09.038251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da93a7e-6598-4b88-a87d-1956e8831c9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:09.054667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da93a7e-6598-4b88-a87d-1956e8831c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2da93a7e-6598-4b88-a87d-1956e8831c9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:09.064321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:09.065305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 14:56:09.066098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:09.069432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 242/479] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:09.074761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:09.198450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79f051-79fc-4b4b-86e3-7862ed707e9f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:09.239722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c07d7a-ebbc-426a-8585-ce5a21af79c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:09.240459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c07d7a-ebbc-426a-8585-ce5a21af79c1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:09.240739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c07d7a-ebbc-426a-8585-ce5a21af79c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:09.241014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c07d7a-ebbc-426a-8585-ce5a21af79c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:09.241818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c07d7a-ebbc-426a-8585-ce5a21af79c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:09.242127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c07d7a-ebbc-426a-8585-ce5a21af79c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:09.243496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c07d7a-ebbc-426a-8585-ce5a21af79c1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:09.253207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 238/480] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:11.065755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd1af2-8bff-4d64-9ce6-ae80728d5278 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.066266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd1af2-8bff-4d64-9ce6-ae80728d5278 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.066378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd1af2-8bff-4d64-9ce6-ae80728d5278 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.067931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 243/481] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:11.073105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31b47e-3b23-4da5-a6c6-6c56de7671cc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.073912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31b47e-3b23-4da5-a6c6-6c56de7671cc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.074397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31b47e-3b23-4da5-a6c6-6c56de7671cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.096646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7e31b47e-3b23-4da5-a6c6-6c56de7671cc None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:11.097757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7e31b47e-3b23-4da5-a6c6-6c56de7671cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:11.180115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f8431-9d43-4945-a6d7-9a38d75f2dee None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.180435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f8431-9d43-4945-a6d7-9a38d75f2dee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.180678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f8431-9d43-4945-a6d7-9a38d75f2dee None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.183331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 244/482] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:11.190498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 239/483] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:11.191679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241fe85-831a-4270-84c9-c9e275911a98 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.191972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241fe85-831a-4270-84c9-c9e275911a98 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.192247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241fe85-831a-4270-84c9-c9e275911a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.213966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31b47e-3b23-4da5-a6c6-6c56de7671cc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.218972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d241fe85-831a-4270-84c9-c9e275911a98 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:11.220192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d241fe85-831a-4270-84c9-c9e275911a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:11.322228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 245/484] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:11.326423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31b47e-3b23-4da5-a6c6-6c56de7671cc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:11.360505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:11.361233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.361777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.361777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.362764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241fe85-831a-4270-84c9-c9e275911a98 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.372867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:11.380715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:11.381609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:11.382907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 240/485] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:11.390186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.478559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241fe85-831a-4270-84c9-c9e275911a98 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:11.494820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c05229-c456-4212-a793-cfff8c366b11 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:11.515088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a739c-819d-45dd-b2ba-ecc11ab1764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:11.515919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a739c-819d-45dd-b2ba-ecc11ab1764a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.516201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a739c-819d-45dd-b2ba-ecc11ab1764a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.520196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a739c-819d-45dd-b2ba-ecc11ab1764a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.526804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a739c-819d-45dd-b2ba-ecc11ab1764a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:11.531886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:11.532555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.532838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.533116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.533874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:11.534179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-eb5a739c-819d-45dd-b2ba-ecc11ab1764a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:11.535549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:11.537731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:11.540342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:11.545124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:11.545217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 246/486] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:11.545217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46ae7d-16f4-4252-a004-70c98fafb05a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.545217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46ae7d-16f4-4252-a004-70c98fafb05a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.545217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46ae7d-16f4-4252-a004-70c98fafb05a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.545217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 247/487] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:11.549229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92761f8-7968-47c7-bcd4-848e58f29629 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.549229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92761f8-7968-47c7-bcd4-848e58f29629 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.549229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92761f8-7968-47c7-bcd4-848e58f29629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.549704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 241/488] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:11.555652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.565288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a92761f8-7968-47c7-bcd4-848e58f29629 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:11.566471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a92761f8-7968-47c7-bcd4-848e58f29629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:11.634285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 248/489] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:11.635066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92761f8-7968-47c7-bcd4-848e58f29629 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.655930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef3665-a142-4b66-ace7-7631b0c7abca None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:11.691402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20069c15-e13b-4458-a07a-5bc2c546e0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:11.692111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20069c15-e13b-4458-a07a-5bc2c546e0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.692395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20069c15-e13b-4458-a07a-5bc2c546e0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.692726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20069c15-e13b-4458-a07a-5bc2c546e0e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.693482 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20069c15-e13b-4458-a07a-5bc2c546e0e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:11.693785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20069c15-e13b-4458-a07a-5bc2c546e0e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:11.695168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20069c15-e13b-4458-a07a-5bc2c546e0e1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:11.711814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 242/490] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:11.711814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573078f6-666d-45ba-952b-a80de4378433 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.711814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573078f6-666d-45ba-952b-a80de4378433 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.711814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573078f6-666d-45ba-952b-a80de4378433 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.711814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 243/491] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:11.711814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573078f6-666d-45ba-952b-a80de4378433 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.749352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92761f8-7968-47c7-bcd4-848e58f29629 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:11.784776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:11.785581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.785861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.786137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.798405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:11.800647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:11.818387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:11.819248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:11.820148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:11.820148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:11.820148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:11.820148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:11.820356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573078f6-666d-45ba-952b-a80de4378433 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:11.820426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 249/492] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:11.820426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.863825 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23406b0c-514c-47b8-95c4-2ad16f60357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:11.864621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23406b0c-514c-47b8-95c4-2ad16f60357f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.864903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23406b0c-514c-47b8-95c4-2ad16f60357f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.865174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23406b0c-514c-47b8-95c4-2ad16f60357f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.879211 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23406b0c-514c-47b8-95c4-2ad16f60357f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-23406b0c-514c-47b8-95c4-2ad16f60357f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:11.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:11.888150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:11.888966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:11.891646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 244/493] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:11.892951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d15cd2-ec08-4251-b214-654b81529664 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.893235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d15cd2-ec08-4251-b214-654b81529664 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.893505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d15cd2-ec08-4251-b214-654b81529664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.909840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c1d15cd2-ec08-4251-b214-654b81529664 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:11.911001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c1d15cd2-ec08-4251-b214-654b81529664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:11.932500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d5f3d-dea9-4638-95a9-8b64fdad7021 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:11.972219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:11.973841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:11.974302 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:11.974852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:11.976865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 245/494] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:11.977700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d15cd2-ec08-4251-b214-654b81529664 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:11.988034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:11.989488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:12.015034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:12.015964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:12.016835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:12.016835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:12.016835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:12.016835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:12.022266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 250/495] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:12.022266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.083464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d15cd2-ec08-4251-b214-654b81529664 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.120032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.120789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.121079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.121355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.121450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2c375-9d4b-40d1-865e-0ffcb9499def None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.122079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:12.122943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.124364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.135620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 246/496] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:12.136473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.156664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.157537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.157958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.158376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.159120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:12.159426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.160840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.168711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 251/497] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:12.170331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.242304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff739ded-d245-4471-819e-f78dda8f13ab None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.267302 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819c11f-6343-4f8d-b929-73e7b91f3dc5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.274154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.274946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.275233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.275513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.285782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:12.293504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:12.295166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:12.296031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 247/498] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:12.296765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.299970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.300603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.300888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.301163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.301879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:12.302181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.303556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.315409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 252/499] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:12.316218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.390766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0f349-439d-4e2f-a093-284cabd26084 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.409543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356df9-7d3f-4b41-9a8e-a4d1eaeb774d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.422202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b175c-e9ee-482b-a281-6877eddc6812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.422980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b175c-e9ee-482b-a281-6877eddc6812 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.423256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b175c-e9ee-482b-a281-6877eddc6812 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.423520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b175c-e9ee-482b-a281-6877eddc6812 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.430694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b175c-e9ee-482b-a281-6877eddc6812 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.432281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b175c-e9ee-482b-a281-6877eddc6812 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.441332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.442102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.442378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.442648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7a1b175c-e9ee-482b-a281-6877eddc6812 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:12.447528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:12.448615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:12.449547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 248/500] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:12.450215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc3e57-7bd5-46b6-a467-c8addd0476e1 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.450871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc3e57-7bd5-46b6-a467-c8addd0476e1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.451323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc3e57-7bd5-46b6-a467-c8addd0476e1 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.452622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 249/501] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:12.454817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.456306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.457916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc3e57-7bd5-46b6-a467-c8addd0476e1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:12.474368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:12.475420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:12.476345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:12.476345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:12.476345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:12.476345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:12.476345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 253/502] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:12.476345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.556138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc3e57-7bd5-46b6-a467-c8addd0476e1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.567544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b9028-ced3-42cb-8054-0d9013c18123 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.589246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.589906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.590448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.590717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.592189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:12.592489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.593857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.601486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 250/503] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:12.602261 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.608969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de98c1a-6e03-48fe-a8f2-d3f06ffeabe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.609757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de98c1a-6e03-48fe-a8f2-d3f06ffeabe3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.610033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de98c1a-6e03-48fe-a8f2-d3f06ffeabe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.610301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de98c1a-6e03-48fe-a8f2-d3f06ffeabe3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.617202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de98c1a-6e03-48fe-a8f2-d3f06ffeabe3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.618629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de98c1a-6e03-48fe-a8f2-d3f06ffeabe3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2de98c1a-6e03-48fe-a8f2-d3f06ffeabe3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:12.634090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:12.635004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:12.635924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 254/504] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:12.636970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200872a1-13da-4aca-a20a-611b93c0286c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.637187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200872a1-13da-4aca-a20a-611b93c0286c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.637465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200872a1-13da-4aca-a20a-611b93c0286c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.654809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-200872a1-13da-4aca-a20a-611b93c0286c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:12.656018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-200872a1-13da-4aca-a20a-611b93c0286c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:12.696501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b3953-8de6-449a-947c-ec03f43dc22d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.722437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 255/505] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:12.723267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200872a1-13da-4aca-a20a-611b93c0286c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.736816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.737483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.737770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.738048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.738892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:12.739202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.740701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.777567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 251/506] 10.222.0.71 () {56 vars in 1137 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:12.778321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.823647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200872a1-13da-4aca-a20a-611b93c0286c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.856204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.857529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.857850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.858079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.858797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:12.859100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.860498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.871791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171a6cc-9713-46cb-8b5d-24f7b97ef84b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:12.874344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 256/507] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:12.876212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.904891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:12.905673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:12.905955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:12.906230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:12.915774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:12.917247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:12.935473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:12.936460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:12.937311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:12.937311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:12.937311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:12.937311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:12.937311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 252/508] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:12.937569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:12.974552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e6cc9-2c36-42c6-b881-c9690ef2cbd9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.006280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.006944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.007226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.007500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.008361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:13.008679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.010100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.031172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b2517-9218-48d1-a8e2-d7701dfda3f2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.050633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 257/509] 10.222.0.71 () {56 vars in 1137 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:13.050633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.067809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.068598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.068880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.069154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.079636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.081120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:13.098424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:13.099898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:13.100750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:13.100750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:13.100750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:13.100750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:13.100750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 253/510] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:13.100750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.140549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1748de-d586-4885-b25a-051bc06e66eb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.173595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.174241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.174524 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.174802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.194152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb05b58-8136-434a-bf22-991f36a89033 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.226703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.227410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.227719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.227992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.234793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.236256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.239133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.240607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:13.251766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:13.252646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:13.253519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 254/511] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:13.254116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.334278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 258/512] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:13.334959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.346823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb66ca-0989-4fd0-8340-d0ef2964f7f2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.378207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.378857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.379141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.379414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.380162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:13.380467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.381850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.392998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 255/513] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:13.393782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.429385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391d3be-cb8f-4cf8-bb3c-46acefc7eb55 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.462917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.463560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.463863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.464136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.486359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699af57-64d2-4fe2-a8a6-a84fbedd7324 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.517810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.518456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.518740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.519011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.519741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:13.520043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.521389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.529261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.531055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.533140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 256/514] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:13.533918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.624452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e5d46-eca8-4009-aeb2-388332a667fa None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.624940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 259/515] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:13.625771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.656976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.657722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.658151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.658151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.658998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:13.659305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.660763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.677736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 257/516] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:13.678528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.719508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed804f5-dc65-4cff-afa6-aa99d916031c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.751960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.752706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.752988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.753264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.762697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.765263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.771780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38f4ee-c88b-4132-aff5-ff2e885bdcf1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:13.782440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:13.783284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:13.784486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:13.784486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:13.784486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:13.784486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:13.784486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 260/517] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:13.784486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.803739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.804531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.804810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.805080 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.812883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.814320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:13.829443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:13.830529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:13.831457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 258/518] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:13.832106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:13.883060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c410ac-6c0d-462d-99c7-2639199898e4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.915801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.916468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.916749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.917026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.917722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.919749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.930904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56634f-4b9d-4bb1-baf7-3e0a2ef1374b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:13.962892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:13.963701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:13.963985 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:13.964262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:13.974669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:13.976132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:13.992957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:13.993952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:13.994885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:13.994885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:13.994885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:13.994885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:13.994885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 259/519] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:13.994885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.007972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 261/520] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:56:13 2022] POST /identity/v3/users => generated 247 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:14.008716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.100621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff37e6-ac31-45ed-852f-3d7ca5e8347f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.118479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4d1dc-1205-406e-9101-6d289f2ef8ee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.136657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.137325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.137610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.137889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.153959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.154796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.155170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.155610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.156469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:14.156903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.158439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.173899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 262/521] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:14.174763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.214962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.217896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.274525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4f328-6066-4688-bd24-4d4cd623890c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.307945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.308618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.308898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.309156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.310612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:14.310914 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.312376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.321491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 260/522] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:14.322502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.330195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 263/523] 10.222.0.71 () {56 vars in 1139 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:14.331080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.443415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7e545-142e-46f0-9432-28abbd029c5c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.474633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11624b3d-4b1c-4fb9-b946-483cef812924 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.479277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.480043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.480390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.480721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.481500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:14.482420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.483887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.499934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 261/524] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:14.499934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.507813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.508611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.508884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.509151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.516251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.517860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:14.532991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:14.534039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:14.535149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:14.538555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 264/525] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:14.539387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.598961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464326-9c1f-40a7-9db2-3e44bce740f7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.632329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.633179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.633456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.633725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.639969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe4bc0-90b0-4262-b11c-ec238e6991de None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.645370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.646837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:14.664049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:14.664894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:14.666305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:14.666305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:14.666305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:14.666305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:14.666305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 262/526] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:14.666305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.677245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.677891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.678171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.678448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.745477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.746930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.762147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd1fb3-6cb4-4f73-a078-9a7505616353 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.795335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.796159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.796440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.796709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.803272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.804806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:14.820414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:14.821310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:14.822147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:14.824029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 263/527] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:14.825088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.845752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 265/528] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:14.846598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.934545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519aac7-57f5-4686-a639-5c851cafe2e3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.944237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b298d0-b203-40e2-9cea-75d29c053149 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:14.967192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.967962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.968241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.968508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.969284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:14.969592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.971005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.978779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:14.979438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:14.979746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:14.980021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:14.980736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:14.981370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:14.982920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:14.993445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 264/529] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:14.993445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:14.994038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 266/530] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:14.994874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.094134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1721b6-9e6d-4ca9-98c4-a0966ca323b4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.094643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cf92f-2b8c-4c08-89d9-9b0a3c119504 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.126587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.126806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.127380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.127492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.127705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.127817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.127996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.128088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.128729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:15.129034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.130469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.138291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.140195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.146619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 265/531] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:15.146619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:15.158532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:15.160073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:15.161217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:15.161217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:15.161217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:15.161217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:15.161217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 267/532] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:15.161217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.253152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b1a14-50c8-43ee-9df9-c16b84ff6234 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.262791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613cc18-2b52-443b-a316-886ae7e930b2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.286352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.287178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.287562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.287963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.288904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=1f2421c1bfb5402da3179e6f3c749374) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:15.289316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.290843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.297161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.299814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 266/533] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/role_assignments?user.id=1f2421c1bfb5402da3179e6f3c749374&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:15.299975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.299975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.299975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.300206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.307468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.309237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:15.326223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:15.333475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 268/534] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:15.334605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.415363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b00ec-90b6-4fbf-802e-26d350f836e4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.439468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9517f50-8033-445b-a85e-34ec4cf321a3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.452425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.453224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.453660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.454031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.454926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=b60d4a75e5b8464a8c7935e7433cf170) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:15.455348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.457875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.476137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 267/535] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/role_assignments?user.id=b60d4a75e5b8464a8c7935e7433cf170&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:15.477303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.483384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.484298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.484727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.485149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.486046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:15.486508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.488725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.501536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 269/536] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:15.506804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.617206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad6767-101a-4a48-94e2-5d3cc6064127 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.627434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c2ec-5cca-4531-b3a0-a2fad5358154 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.655085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.655941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.656320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.656694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.657598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:15.658037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.659599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.673893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 268/537] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:15.676440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.677268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.677552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.677824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.680673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.686228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.687842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:15.704740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:15.705649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:15.706485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 270/538] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:15.711337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:15.797593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee9353-d1e0-4cce-b745-7acf0c2a95fc None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.846396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.847303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.847827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.848301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.849355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=3d58a6e2eac047ad8e88a1f973ecc697) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:15.849882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.851520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.862593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 269/539] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/role_assignments?user.id=3d58a6e2eac047ad8e88a1f973ecc697&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:15.879711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bafe4-7e2b-41ae-8cb1-814bdf5d8a48 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:15.923994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40720604-a9a0-43c2-a147-a2e357d49277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:15.924602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40720604-a9a0-43c2-a147-a2e357d49277 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:15.924884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40720604-a9a0-43c2-a147-a2e357d49277 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:15.925164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40720604-a9a0-43c2-a147-a2e357d49277 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:15.925967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40720604-a9a0-43c2-a147-a2e357d49277 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:15.926275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40720604-a9a0-43c2-a147-a2e357d49277 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:15.927758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40720604-a9a0-43c2-a147-a2e357d49277 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:15.952614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 271/540] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:15.958196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:16.111209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a04535-4bff-4b9a-9b5e-c3bc19484e93 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:16.147805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8dd6-0873-4508-916d-a07d766ce5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:16.148568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8dd6-0873-4508-916d-a07d766ce5df None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:16.148747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8dd6-0873-4508-916d-a07d766ce5df None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:16.149019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8dd6-0873-4508-916d-a07d766ce5df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:16.149862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8dd6-0873-4508-916d-a07d766ce5df None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=76b2a2d381a14f5fb04e721905b36ace) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:16.150171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8dd6-0873-4508-916d-a07d766ce5df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:16.151656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8dd6-0873-4508-916d-a07d766ce5df None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:16.158961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 270/541] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:15 2022] GET /identity/v3/role_assignments?user.id=76b2a2d381a14f5fb04e721905b36ace&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:16.486626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aca805-d2a0-43fb-a03c-dc98ee64f3d3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:16.487135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aca805-d2a0-43fb-a03c-dc98ee64f3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:16.487243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aca805-d2a0-43fb-a03c-dc98ee64f3d3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:16.488812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 272/542] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:16.506150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2737f4-4670-43f0-9ca3-1060929234e4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:16.506471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2737f4-4670-43f0-9ca3-1060929234e4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:16.506711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2737f4-4670-43f0-9ca3-1060929234e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:16.532680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1d2737f4-4670-43f0-9ca3-1060929234e4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:16.533909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1d2737f4-4670-43f0-9ca3-1060929234e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:16.621051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 271/543] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:16.656018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aca805-d2a0-43fb-a03c-dc98ee64f3d3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:16.785338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aca805-d2a0-43fb-a03c-dc98ee64f3d3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:16.834338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:16.835538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:16.836062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:16.836536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:16.860648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:16.868821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:16.869700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:16.870490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:16.874254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 273/544] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:16.880754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2737f4-4670-43f0-9ca3-1060929234e4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:17.042489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2737f4-4670-43f0-9ca3-1060929234e4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:17.092352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:17.093036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:17.093321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:17.093597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:17.094348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:17.094653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:17.096056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:17.104584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 272/545] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:17.110104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:17.289948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0347e2-4384-49ea-91ae-b2a11cfae89f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:17.332360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:17.333084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:17.333368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:17.333645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:17.348894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:17.350347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:17.370033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:17.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:17.371862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:17.371862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:17.371862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:17.371862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:17.371862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 274/546] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:17.376972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:17.507373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195eda69-9f65-4ac3-86ab-0ea9851c1598 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:17.553659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:17.554320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:17.554599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:17.554867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:17.555590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:17.555916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:17.557271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:17.574238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 273/547] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:17.579845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:17.697252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413b3d4-625a-4060-8465-d1537afe2eb6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:17.735235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:17.736061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:17.736339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:17.736608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:17.747266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:17.748998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:17.766056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:17.768197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:17.770375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 275/548] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:17.776716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:17.931713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da3a43-7f70-4727-812d-49493df8e289 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:17.981692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f6f21-49e3-4421-82fb-60f1945e692b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:17.983515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f6f21-49e3-4421-82fb-60f1945e692b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:17.984021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f6f21-49e3-4421-82fb-60f1945e692b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:17.984496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f6f21-49e3-4421-82fb-60f1945e692b None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:17.985559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f6f21-49e3-4421-82fb-60f1945e692b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:17.986077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f6f21-49e3-4421-82fb-60f1945e692b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:17.987720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f6f21-49e3-4421-82fb-60f1945e692b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:18.017881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 274/549] 10.222.0.71 () {56 vars in 1135 bytes} [Mon May 16 14:56:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:18.023497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:18.072274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd1a94-ec36-4188-923e-148780d1b634 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.072916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd1a94-ec36-4188-923e-148780d1b634 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.073417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd1a94-ec36-4188-923e-148780d1b634 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.074976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 275/550] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:18.080783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d8bca-c901-462f-bd7c-aa0fdee057f5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.081335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d8bca-c901-462f-bd7c-aa0fdee057f5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.081832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d8bca-c901-462f-bd7c-aa0fdee057f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.112707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-909d8bca-c901-462f-bd7c-aa0fdee057f5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:18.114253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-909d8bca-c901-462f-bd7c-aa0fdee057f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:18.151720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13345b-b615-44a4-b57e-723a1cad55dd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:18.177454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:18.178109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.178390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.178660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.237193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 276/551] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:18.251289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:18.252800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:18.272566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d8bca-c901-462f-bd7c-aa0fdee057f5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:18.354976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 276/552] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:18.360072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:18.497148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d8bca-c901-462f-bd7c-aa0fdee057f5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:18.564659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f5b57-0cf7-4b8c-af26-fc526e080013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:18.565640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5b57-0cf7-4b8c-af26-fc526e080013 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.566337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5b57-0cf7-4b8c-af26-fc526e080013 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.566623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5b57-0cf7-4b8c-af26-fc526e080013 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f5b57-0cf7-4b8c-af26-fc526e080013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:18.585755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba0447-629b-4bcc-a2fb-29cfbe316cec None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5e6f5b57-0cf7-4b8c-af26-fc526e080013 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:18.593330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:18.594469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:18.595460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 277/553] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:18.599665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf019-a5b2-4575-b461-ff5fc72c0807 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.599665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf019-a5b2-4575-b461-ff5fc72c0807 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.599665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf019-a5b2-4575-b461-ff5fc72c0807 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.600108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 278/554] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:18.600895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf019-a5b2-4575-b461-ff5fc72c0807 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:18.631188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76efbf-0f21-45f5-a78d-87b84ea85084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:18.632194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76efbf-0f21-45f5-a78d-87b84ea85084 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.632539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76efbf-0f21-45f5-a78d-87b84ea85084 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.632925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76efbf-0f21-45f5-a78d-87b84ea85084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.645734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76efbf-0f21-45f5-a78d-87b84ea85084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:18.647424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76efbf-0f21-45f5-a78d-87b84ea85084 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4a76efbf-0f21-45f5-a78d-87b84ea85084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:18.669649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:18.671754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:18.673297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:18.673297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:18.673297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:18.673297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:18.673297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 277/555] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:18.677539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2b730-9a5f-491c-9044-534bc4ce095a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.677847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2b730-9a5f-491c-9044-534bc4ce095a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.678196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2b730-9a5f-491c-9044-534bc4ce095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.695000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-62a2b730-9a5f-491c-9044-534bc4ce095a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:18.696354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-62a2b730-9a5f-491c-9044-534bc4ce095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:18.726985 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf019-a5b2-4575-b461-ff5fc72c0807 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:18.761381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb8899-ac65-4ddd-b8bf-395355272d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:18.761971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb8899-ac65-4ddd-b8bf-395355272d81 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.762253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb8899-ac65-4ddd-b8bf-395355272d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.762519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb8899-ac65-4ddd-b8bf-395355272d81 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.764021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb8899-ac65-4ddd-b8bf-395355272d81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:18.764411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb8899-ac65-4ddd-b8bf-395355272d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:18.766047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb8899-ac65-4ddd-b8bf-395355272d81 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:18.766167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 278/556] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:18.767091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2b730-9a5f-491c-9044-534bc4ce095a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:18.774867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 279/557] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:18.776249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5e413-266a-40b5-9db1-3e185c374bab None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.776535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5e413-266a-40b5-9db1-3e185c374bab None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.776808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5e413-266a-40b5-9db1-3e185c374bab None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.778279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 280/558] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:18.783772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5e413-266a-40b5-9db1-3e185c374bab None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:18.868354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2b730-9a5f-491c-9044-534bc4ce095a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:18.883933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5e413-266a-40b5-9db1-3e185c374bab None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:18.900724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c483a-e128-4911-bd53-597f2a75fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:18.901383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c483a-e128-4911-bd53-597f2a75fa38 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.901661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c483a-e128-4911-bd53-597f2a75fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.901928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c483a-e128-4911-bd53-597f2a75fa38 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.902634 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c483a-e128-4911-bd53-597f2a75fa38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:18.902934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c483a-e128-4911-bd53-597f2a75fa38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:18.905048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c483a-e128-4911-bd53-597f2a75fa38 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:18.915635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 279/559] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:18.915757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:18.916485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.916771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.916926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0787c-cec4-41f0-991b-1f49c4956b41 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:18.917038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.917199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0787c-cec4-41f0-991b-1f49c4956b41 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:18.917474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0787c-cec4-41f0-991b-1f49c4956b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:18.930035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:18.931486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:18.934801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e4d0787c-cec4-41f0-991b-1f49c4956b41 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:18.936029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e4d0787c-cec4-41f0-991b-1f49c4956b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:18.948283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:18.949210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:18.950057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:18.950057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:18.950057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:18.950057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:18.950057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 281/560] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:18.950342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.002806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 280/561] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:19.004261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c6fd8-c038-4df9-a2f5-434481b0a68e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.004549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c6fd8-c038-4df9-a2f5-434481b0a68e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.004819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c6fd8-c038-4df9-a2f5-434481b0a68e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.006058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 281/562] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:19.006736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c6fd8-c038-4df9-a2f5-434481b0a68e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.051881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b830ec-d62d-4778-8676-6009d3697585 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.085693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.086481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.086766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.087043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.097194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:19.104935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c6fd8-c038-4df9-a2f5-434481b0a68e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:19.105148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:19.106203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 282/563] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:19.107335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.141818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e89c0-2d56-42a1-8888-2457d7e36437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.142613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e89c0-2d56-42a1-8888-2457d7e36437 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.142892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e89c0-2d56-42a1-8888-2457d7e36437 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.143164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e89c0-2d56-42a1-8888-2457d7e36437 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.150011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e89c0-2d56-42a1-8888-2457d7e36437 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.151553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e89c0-2d56-42a1-8888-2457d7e36437 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c81e89c0-2d56-42a1-8888-2457d7e36437 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:19.165017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:19.166037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:19.166942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 282/564] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:19.167788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06eaa3-6053-43d8-832e-2e2bd378d8a5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.167788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06eaa3-6053-43d8-832e-2e2bd378d8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.167927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06eaa3-6053-43d8-832e-2e2bd378d8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.184027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ca06eaa3-6053-43d8-832e-2e2bd378d8a5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:19.185312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ca06eaa3-6053-43d8-832e-2e2bd378d8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:19.204189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43e705-714a-4373-81b2-8ce53d913877 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.236199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.236848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.237128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.237398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.238111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:19.238412 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.239788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.249035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 283/565] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:19.249905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06eaa3-6053-43d8-832e-2e2bd378d8a5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.256151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 283/566] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:19.256151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.352021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d24cb-9b8f-44ec-ad96-b4a44f71a338 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.353119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06eaa3-6053-43d8-832e-2e2bd378d8a5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.384914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.385566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.385776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.385901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.386100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.386567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.386833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:19.386928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.387106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.387201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.388510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.395985 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 284/567] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:19.396783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.397248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:19.405310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:19.406373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:19.407311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 284/568] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:19.408459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.494665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a644dee-17b7-4600-9f74-344e3baffc0a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.504075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc229ac0-8760-469d-ad76-9cb15bbebaab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.525970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.526787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.527066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.527335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.528154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:19.528458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.530443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.536464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.537256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.537538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.537814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.541826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 285/569] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:19.542519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.545156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.546647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:19.561505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:19.562581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 285/570] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:19.563503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.637299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941137c-afb6-4c43-b700-44800b2bfe36 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.658276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde814f8-4bb4-4637-ba53-66c97409e7b6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.668831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.670241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.670524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.670794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.680328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:19.688048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:19.689040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 286/571] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:19.690629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.695083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.695982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.696326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.696607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.707219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.708682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:19.726072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:19.726987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:19.728145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:19.728145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:19.728145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:19.728145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:19.728145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 286/572] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:19.728145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.787188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d3f8e-ffb5-4f6b-9dce-ea964feef332 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.818691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.819350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.819652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.819932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.820671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:19.820977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.822332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.826614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba893f-e20b-4000-a57e-dad7ee2c3d5b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.830271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 287/573] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:19.831045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.860698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.861353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.861634 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.861910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.862732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=918d67038f9c4d0fa4d2456409245774) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:19.863044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.864508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.871030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 287/574] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/role_assignments?user.id=918d67038f9c4d0fa4d2456409245774&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:19.871926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:19.933292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b14ac-89e4-4e0f-8e8f-6fa5a8cba104 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.969880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:19.970586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:19.970853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:19.971126 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:19.972026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:19.972335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:19.973803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:19.975282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749a741-fa46-476a-b05d-12ddf33b8ae2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:19.997544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 288/575] 10.222.0.71 () {56 vars in 1137 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:19.997544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.009948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.010633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.010915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.011187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.012015 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:20.012318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.014262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.027514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 288/576] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:20.027514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.107028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733e661-1e11-4dd8-a082-265c60388871 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.123709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbd517-948a-4691-ae11-edefe715c75a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.142722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.143386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.143690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.144024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.144745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:20.145101 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.146542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.156258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.157052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.157333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.157605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.158433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 289/577] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:20.159314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.173340 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.174782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:20.192595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:20.193521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:20.194446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:20.194446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:20.194446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:20.194446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:20.194446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 289/578] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:20.195270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.260523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979765b-1d4c-4b59-bd8c-b1370e98b42b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.295750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.296583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.296860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.297136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.297520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e5ffa-c506-4570-b526-0ab9c69062aa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.309523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.311587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:20.328755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:20.340040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:20.341415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:20.341415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:20.341415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:20.341415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:20.341415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 290/579] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:20.341415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.343139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.343139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.343139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.343139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.352257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.353757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:20.373722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:20.375789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:20.376675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:20.376675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:20.376675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:20.376675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:20.379499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 290/580] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:20.380369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.431778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5fab8-437a-468d-bb3d-36835ed06efd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.464877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.465933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.466214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.467039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.475550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.477011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.491836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4ff50-f778-48bc-b9c4-1f6759f878d5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:20.492064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:20.493206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 16 14:56:20.494497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:20.498357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 291/581] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:20.499278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.529713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.530484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.530764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.531038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.531793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:20.532100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.533484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.550707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 291/582] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:20.550707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.601129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183da61-4d8e-4d45-b4ca-ffd2948f8059 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.644380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.645053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.645336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.645435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d732cd-0f29-4ca0-9b06-7e4083f9dd31 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.645620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.646334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:20.646640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.648072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.659771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 292/583] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:20.660591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.679584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.680282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.680563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.680834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.681557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:20.681863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.683221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.699841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 292/584] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:20.699841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.773629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6f6d-6d6e-4e79-a8cd-1254c987673b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.793015 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8f5d3-5563-4a81-93e2-d9ac5f87a1be None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.806273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.806839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.807120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.807398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.808274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:20.808584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.810023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.826765 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.827739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.832485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.832651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 293/585] 10.222.0.71 () {56 vars in 1137 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:20.832753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.832845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.836560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.838097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:20.852853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:20.853818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 16 14:56:20.854735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:20.859296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 293/586] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:20.860067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:20.935458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278ce49-313d-4c93-97ff-aab6d15283c9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.955486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1458fd8-f7a0-470a-ab58-f5f06516c3c3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:20.968874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.969699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.969983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.970259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.977675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:20.979152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:20.990642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:20.991516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:20.991888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:20.992226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:20.993707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:20.994718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 294/587] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:21.000743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.001416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.002718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:21.017183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:21.018396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:21.019362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 294/588] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:21.020351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.104168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427db66-6828-4f84-989a-ed1e4fc256a9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.118989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7295-342c-49ad-a834-85b4d1c1d56f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.137017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.137818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.138100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.138407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.149346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.150788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.152499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.153173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.153453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.153725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.154764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:21.155067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.156508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:21.167662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:21.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:21.169452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:21.169452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:21.169452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:21.169452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:21.173122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 295/589] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:21.173122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.180289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 295/590] 10.222.0.71 () {56 vars in 1139 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:21.180289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.270092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb91daa-9374-4765-aeea-93dc5499c4d1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.277336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51942e79-11d8-41cc-941d-be84fba32323 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.303597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.304276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.304559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.304841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.305626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:21.305938 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.307354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.309876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.317923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.317923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.317923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.317923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:21.317923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.317923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.320274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 296/591] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:21.321295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.337035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 296/592] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:21.337861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.417716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b645579-df41-4875-929c-ef3ceeae5729 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.437442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d132a13-f4ce-4e4e-b800-d886d91c6704 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.449891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.450802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.452115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.452398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.453121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:21.453424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.454768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.465639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 297/593] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:21.467257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.479380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.480688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.481117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.481686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.493718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.496092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:21.518346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:21.519351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:21.520269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:21.520269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:21.520269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:21.520269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:21.520269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 297/594] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:21.521068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.567804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0a29f-d812-4852-a193-570a9246f3dc None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.600481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.601491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.602047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.602047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/1f2421c1bfb5402da3179e6f3c749374/roles/29b60e8912a74b2dafcb91fbb14ee519` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.616254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=1f2421c1bfb5402da3179e6f3c749374, role_id=29b60e8912a74b2dafcb91fbb14ee519)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.617996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.619537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094d783-6f81-4587-be8a-31ca0189a9c5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.652894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.653690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.653973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.654367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.664143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.665604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:21.689961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:21.691450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:21.692432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:21.692432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:21.692432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:21.692432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:21.692432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 298/595] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:21.692432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.713436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 298/596] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:21 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/1f2421c1bfb5402da3179e6f3c749374/roles/29b60e8912a74b2dafcb91fbb14ee519 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:56:21.714523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.792293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07ebcf-701b-4972-9942-b7175f0f5c16 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.832314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e5e68-f52a-434b-b0b5-5b40ca7b892c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:21.833999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.835090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.835550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.836070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.844693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.847190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:21.862176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:21.863176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:21.864080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:21.866301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 299/597] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:21.867302 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.876066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:21.876757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:21.877041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:21.877318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:21.878087 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:21.878747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:21.879828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:21.897483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 299/598] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:21.897483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:21.985659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b88da1-bfc2-43a4-b8cf-46085ceb75d5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.010965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156db4e-75d1-479b-a57d-f7dd24e78ea8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.020638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.021318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.021598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.021873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.022597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:22.022910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.024302 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.036722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 300/599] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:22.037619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.047141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.048010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.048389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.048758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.049648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:22.050047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.051558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.068699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 300/600] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:22.069837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.140671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5e5e-f983-466c-8aef-1ca1a9cc8d60 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.173650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.174532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.174816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.174982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110692a-78b7-4ce1-9517-a3774ad3f28b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.175086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.181768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.183207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:22.197449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:22.198401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:22.199311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 301/601] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:22.199833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.212413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ccaa6-87ec-4ef8-a40a-e531a33343a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.213210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ccaa6-87ec-4ef8-a40a-e531a33343a6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.213488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ccaa6-87ec-4ef8-a40a-e531a33343a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.213756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ccaa6-87ec-4ef8-a40a-e531a33343a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.221383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ccaa6-87ec-4ef8-a40a-e531a33343a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.222814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ccaa6-87ec-4ef8-a40a-e531a33343a6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-fb0ccaa6-87ec-4ef8-a40a-e531a33343a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:22.236956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:22.238057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:22.239129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 301/602] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:22.244088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5114001-08ac-451a-9529-ee42f2445625 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.244088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5114001-08ac-451a-9529-ee42f2445625 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.244088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5114001-08ac-451a-9529-ee42f2445625 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.244088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 302/603] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:22.244088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5114001-08ac-451a-9529-ee42f2445625 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.303262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45463d-ab81-462b-a493-01c03e866932 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.337470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a111559-a24c-48aa-bbb5-3a4b2c4805db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.338367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a111559-a24c-48aa-bbb5-3a4b2c4805db None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.338705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a111559-a24c-48aa-bbb5-3a4b2c4805db None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.338977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a111559-a24c-48aa-bbb5-3a4b2c4805db None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/b60d4a75e5b8464a8c7935e7433cf170/roles/29b60e8912a74b2dafcb91fbb14ee519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.347387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5114001-08ac-451a-9529-ee42f2445625 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.354076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a111559-a24c-48aa-bbb5-3a4b2c4805db None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=b60d4a75e5b8464a8c7935e7433cf170, role_id=29b60e8912a74b2dafcb91fbb14ee519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.355528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a111559-a24c-48aa-bbb5-3a4b2c4805db None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.381408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.382078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.382363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.382638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.383423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:22.383762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.385186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.396644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 303/604] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:22.397434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.437693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 302/605] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:22 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/b60d4a75e5b8464a8c7935e7433cf170/roles/29b60e8912a74b2dafcb91fbb14ee519 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:56:22.439295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd436b4-d1ec-4c17-a3e6-a7cfed01bccd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.439658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd436b4-d1ec-4c17-a3e6-a7cfed01bccd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.440003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd436b4-d1ec-4c17-a3e6-a7cfed01bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.458851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6dd436b4-d1ec-4c17-a3e6-a7cfed01bccd None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:22.460156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6dd436b4-d1ec-4c17-a3e6-a7cfed01bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:22.535763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1518bb-1405-450e-8904-016dc0e6a8e5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.542763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 303/606] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:22.543914 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd436b4-d1ec-4c17-a3e6-a7cfed01bccd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.578971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.580011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.580664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.581157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.582495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:22.582981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.585157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.601803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 304/607] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:22.603464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.674636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd436b4-d1ec-4c17-a3e6-a7cfed01bccd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.710550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.711439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.711749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.712019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/76b2a2d381a14f5fb04e721905b36ace/roles/29b60e8912a74b2dafcb91fbb14ee519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.728693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2ae45-61ad-48d3-ac8c-d5f5d8891b1b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.733254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=76b2a2d381a14f5fb04e721905b36ace, role_id=29b60e8912a74b2dafcb91fbb14ee519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:22.736483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:22.767679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.768557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.768907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.769254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:22.779379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:22.791578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:22.792603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 305/608] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:22.793454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.817906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 304/609] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:22 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/76b2a2d381a14f5fb04e721905b36ace/roles/29b60e8912a74b2dafcb91fbb14ee519 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:56:22.819170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:22.916710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5ebf9-8dfb-48fa-a9c6-2aad9ca863da None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.958848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81387263-fc81-4c4f-92d1-f5d9d4e1f8ef None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:22.966297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406245d6-8357-4873-992a-558521bfe145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:22.967254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406245d6-8357-4873-992a-558521bfe145 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:22.967624 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406245d6-8357-4873-992a-558521bfe145 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:22.968018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406245d6-8357-4873-992a-558521bfe145 None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/3d58a6e2eac047ad8e88a1f973ecc697/roles/29b60e8912a74b2dafcb91fbb14ee519` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:23.001264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406245d6-8357-4873-992a-558521bfe145 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=3d58a6e2eac047ad8e88a1f973ecc697, role_id=29b60e8912a74b2dafcb91fbb14ee519)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:23.002864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406245d6-8357-4873-992a-558521bfe145 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:23.012619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:23.013775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:23.014269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:23.014799 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:23.015951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:23.026112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:23.026112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:23.031254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 305/610] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:23.036709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:23.079392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 306/611] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:22 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/3d58a6e2eac047ad8e88a1f973ecc697/roles/29b60e8912a74b2dafcb91fbb14ee519 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:56:23.167997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e8b28-419d-435c-9d8d-61be83118944 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:23.213941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:23.215371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:23.216169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:23.216701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:23.229959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:23.231700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:23.255349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:23.256359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:23.258106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:23.258106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:23.258106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:23.258106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:23.258106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 306/612] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:23.262634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406245d6-8357-4873-992a-558521bfe145 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:23.402451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406245d6-8357-4873-992a-558521bfe145 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:23.443998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:23.444677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:23.444960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:23.445232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:23.445971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:23.446271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:23.447660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:23.460489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 307/613] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:23.466463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:23.598435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e965220-1e03-44fa-ba59-fcdccba4508d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:23.643139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:23.643973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:23.644252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:23.644524 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:23.653005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:23.654485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:23.674992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:23.677353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:23.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 307/614] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:23.683652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:23.843055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c55cf-b315-4bc0-b26e-e7a0d4387e75 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:23.883544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:23.884246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:23.884530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:23.884810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:23.885659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:23.885969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:23.887424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:23.907963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 308/615] 10.222.0.71 () {56 vars in 1135 bytes} [Mon May 16 14:56:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:23.916758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:24.043736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f30f58-f72c-4df3-9e1f-4079c636d55a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:24.083099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:24.083929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:24.084211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:24.084483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:24.095727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:24.097180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:24.124802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:24.126982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:24.129233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:24.129233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:24.129233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:24.129233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:24.129233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 308/616] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:24.140817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:24.315601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f34fb-c6bb-4c24-a102-55d45cb141c0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:24.364329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80276747-d656-4d3e-b6b5-bcd6133787c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:24.365383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276747-d656-4d3e-b6b5-bcd6133787c5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:24.365872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276747-d656-4d3e-b6b5-bcd6133787c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:24.366376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276747-d656-4d3e-b6b5-bcd6133787c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:24.367308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80276747-d656-4d3e-b6b5-bcd6133787c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:24.367836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80276747-d656-4d3e-b6b5-bcd6133787c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:24.369440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80276747-d656-4d3e-b6b5-bcd6133787c5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:24.382498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 309/617] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:24.387577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:24.415029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74856e6b-329f-434e-a634-c3305cbd9ba7 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:24.415698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74856e6b-329f-434e-a634-c3305cbd9ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:24.416195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74856e6b-329f-434e-a634-c3305cbd9ba7 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:24.417753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 310/618] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:24.423391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884a4a-e03d-4c15-9423-747578a3b8d7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:24.423987 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884a4a-e03d-4c15-9423-747578a3b8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:24.424475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884a4a-e03d-4c15-9423-747578a3b8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:24.455917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ba884a4a-e03d-4c15-9423-747578a3b8d7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:24.457432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ba884a4a-e03d-4c15-9423-747578a3b8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:24.507843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190e1be-e641-4277-871c-76c1463baf96 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:24.535177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:24.536131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:24.536421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:24.536693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:24.543571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:24.545591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:24.569179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:24.570143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:24.571427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 309/619] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:24.591739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:24.614723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 311/620] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:24.669502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba884a4a-e03d-4c15-9423-747578a3b8d7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:24.744511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6f221-e537-411a-aca0-916f3ae75354 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:24.819899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba884a4a-e03d-4c15-9423-747578a3b8d7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:24.831228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:24.831990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:24.832273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:24.832552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:24.833355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:24.833666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:24.835121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:24.854526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:24.855322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:24.855597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:24.856371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:24.867093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 310/621] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:24.868026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:24.880218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:24.881392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:24.883527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:24.885595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 312/622] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:24.892489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:24.986455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246983f-fb30-424c-8f75-fa69edd5de7f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:25.024495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b3564-ad7c-443c-a54c-1d67dca8f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:25.025384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3564-ad7c-443c-a54c-1d67dca8f9a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.025665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3564-ad7c-443c-a54c-1d67dca8f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.025938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3564-ad7c-443c-a54c-1d67dca8f9a1 None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/918d67038f9c4d0fa4d2456409245774/roles/29b60e8912a74b2dafcb91fbb14ee519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.063342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b3564-ad7c-443c-a54c-1d67dca8f9a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=918d67038f9c4d0fa4d2456409245774, role_id=29b60e8912a74b2dafcb91fbb14ee519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:25.067805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b3564-ad7c-443c-a54c-1d67dca8f9a1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:25.101368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2f1b9-5ba9-426c-8b78-d2e051808c51 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:25.128641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:25.130070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.130070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.130070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.130590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:25.131659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:25.133946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:25.141738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 313/623] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:25.146982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:25.171168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 311/624] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:24 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/918d67038f9c4d0fa4d2456409245774/roles/29b60e8912a74b2dafcb91fbb14ee519 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:56:25.260126 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b04fa-f544-4343-8d7b-4b2785d5ea20 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:25.296368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:25.297174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.297457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.297729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.308868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:25.310364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:25.324666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ef624-aefb-4e49-885b-7a64787c5159 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.324993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ef624-aefb-4e49-885b-7a64787c5159 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.325206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ef624-aefb-4e49-885b-7a64787c5159 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.326989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 312/625] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:25.329867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:25.332027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:25.334289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:25.334289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:25.334289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:25.334289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:25.334289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 314/626] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:25.334955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4685-19c2-44da-becd-20bd0ff32886 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.334955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4685-19c2-44da-becd-20bd0ff32886 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.334955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4685-19c2-44da-becd-20bd0ff32886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.339671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:25.367933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8d5b4685-19c2-44da-becd-20bd0ff32886 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:25.369071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d5b4685-19c2-44da-becd-20bd0ff32886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:25.488740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 313/627] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:25.499185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc3f4f-f08c-4330-bf9a-d71794d13bcc None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:25.519828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b4685-19c2-44da-becd-20bd0ff32886 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:25.539347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:25.540263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.540548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.541200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.541966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:25.542273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:25.543764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:25.566686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 315/628] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:25.577392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:25.656766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b4685-19c2-44da-becd-20bd0ff32886 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:25.705845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b3dbe-7581-4982-8d75-62780a29cbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:25.706660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3dbe-7581-4982-8d75-62780a29cbfa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.706936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3dbe-7581-4982-8d75-62780a29cbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.707207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3dbe-7581-4982-8d75-62780a29cbfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.720991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90465d9f-3896-4483-b5fb-ae6e9ae95fbd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:25.721652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b3dbe-7581-4982-8d75-62780a29cbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-294b3dbe-7581-4982-8d75-62780a29cbfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:25.729664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:25.730666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 314/629] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:25.732681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d0438-44ed-4bd1-8122-e27f402b2953 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.734477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d0438-44ed-4bd1-8122-e27f402b2953 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.734477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d0438-44ed-4bd1-8122-e27f402b2953 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.738934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 315/630] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:25.739947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d0438-44ed-4bd1-8122-e27f402b2953 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:25.765910 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef02ce-f79c-44c7-9767-495a76e4f856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:25.767285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef02ce-f79c-44c7-9767-495a76e4f856 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.768016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef02ce-f79c-44c7-9767-495a76e4f856 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.768303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef02ce-f79c-44c7-9767-495a76e4f856 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.778188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef02ce-f79c-44c7-9767-495a76e4f856 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:25.779694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef02ce-f79c-44c7-9767-495a76e4f856 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a3ef02ce-f79c-44c7-9767-495a76e4f856 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:25.798200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:25.800033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 316/631] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:25.801056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5f70-bed5-4ca9-9ca2-8b3ab037d168 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.801707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5f70-bed5-4ca9-9ca2-8b3ab037d168 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.801707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5f70-bed5-4ca9-9ca2-8b3ab037d168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.821737 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-72cf5f70-bed5-4ca9-9ca2-8b3ab037d168 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:25.822931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-72cf5f70-bed5-4ca9-9ca2-8b3ab037d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:25.871892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d0438-44ed-4bd1-8122-e27f402b2953 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:25.903680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 317/632] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:25.903680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5f70-bed5-4ca9-9ca2-8b3ab037d168 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:25.924406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2da7a-8875-43a9-b914-a292282e8991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:25.925550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2da7a-8875-43a9-b914-a292282e8991 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.926002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2da7a-8875-43a9-b914-a292282e8991 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.926425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2da7a-8875-43a9-b914-a292282e8991 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.927432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2da7a-8875-43a9-b914-a292282e8991 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:25.927957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2da7a-8875-43a9-b914-a292282e8991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:25.929695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2da7a-8875-43a9-b914-a292282e8991 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:25.939481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 316/633] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:25.940853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc103b-9feb-482b-b8ce-683de6da5297 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:25.941150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc103b-9feb-482b-b8ce-683de6da5297 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:25.941424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc103b-9feb-482b-b8ce-683de6da5297 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:25.944711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 317/634] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:25.944711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc103b-9feb-482b-b8ce-683de6da5297 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.003106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5f70-bed5-4ca9-9ca2-8b3ab037d168 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.038040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.038714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.038991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.039265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.040138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:26.040443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.041856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.046802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc103b-9feb-482b-b8ce-683de6da5297 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.059805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 318/635] 10.222.0.71 () {56 vars in 1137 bytes} [Mon May 16 14:56:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:26.060470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.080917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb585c8-0563-42b0-9cb1-0dffdf872909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.081718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb585c8-0563-42b0-9cb1-0dffdf872909 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.082002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb585c8-0563-42b0-9cb1-0dffdf872909 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.082279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb585c8-0563-42b0-9cb1-0dffdf872909 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.092606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb585c8-0563-42b0-9cb1-0dffdf872909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bdb585c8-0563-42b0-9cb1-0dffdf872909 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:26.100404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:26.101339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:26.102188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 318/636] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:26.103431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf596cd5-2877-47f4-ba30-ca8f2fc62d49 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.103431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf596cd5-2877-47f4-ba30-ca8f2fc62d49 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.103723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf596cd5-2877-47f4-ba30-ca8f2fc62d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.120516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bf596cd5-2877-47f4-ba30-ca8f2fc62d49 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:26.121710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bf596cd5-2877-47f4-ba30-ca8f2fc62d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:26.166018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f539-9836-43e9-a426-5dce60e3d015 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.191769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 319/637] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:26.191769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf596cd5-2877-47f4-ba30-ca8f2fc62d49 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.209420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.210426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.210873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.211286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.222576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.224050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.296459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf596cd5-2877-47f4-ba30-ca8f2fc62d49 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.332047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.332746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.332998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.333273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:26.363391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:26.364583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:26.365502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:26.365502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:26.365502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:26.365502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:26.365502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 319/638] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:26.365502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.402671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.405204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.466507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fccfb6-cbd2-487e-bfff-795468f5d92f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.500631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.501284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.506179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.506344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 320/639] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:26.506450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.506450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:26.506617 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.506702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.506702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.513720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 320/640] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:26.513720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.601998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eab9f2-6a49-4630-bec6-bec58867cc1b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.621442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a5945-a83f-42a3-8009-bcfbe970d687 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.637552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.638647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.639089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.639382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.650604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:26.655271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.656142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.656575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.657017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.657903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:26.658357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:26.659393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:26.660373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:26.661211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:26.661855 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.663942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 321/641] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:26.664994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.677204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 321/642] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:26.677204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.768518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da43c8-4f36-4a31-9ad3-d35052c3ddc7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.774247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4641f-5034-4913-8f22-45b8bad1cb3b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.801392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.802183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.802461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.802736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.807372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.808184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.808467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.808736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.813250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.814692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.818777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.820238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:26.835163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:26.836257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:26.837189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:26.837189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:26.837189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:26.837189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:26.838322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:26.845416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:26.848657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:26.848657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:26.848657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:26.848657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:26.848657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 322/643] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:26.849024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 322/644] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:26.849024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.849187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:26.940578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfbda3-86d4-4e06-ada3-0cd1ea117dda None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.947791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656ea31-6c12-48a4-b59e-6d73949adea3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:26.972350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.973149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.973433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.973699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] PATH_INFO: `/v3/users/glance` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.981057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.981278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:26.982161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:26.982511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:26.983883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:26.984333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:26.985262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:26.985798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:26.987267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.000062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 323/645] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:27.000062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:27.000293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:27.001199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 323/646] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:27.002148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.098572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1d536-7a54-4832-81a7-d82520d94a55 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.100225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd7b95-e05c-400c-84b6-b373901b6bee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.132269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.133033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.133315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.133590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.134144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.134349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:27.134618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.134801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.135078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.135344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.136003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.136193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:27.136493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.137832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.152357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 324/647] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:27.152357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.155384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 324/648] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:27.159733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.245830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c4610-fe90-4a25-a04a-e46129a16312 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.256823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb9d65-e8df-484b-a6dd-2fc93ef4f20f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.280468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.281254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.281538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.281813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.290829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.291496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.291807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.292077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.292817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.292918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:27.293196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.294270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.294651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.312415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 325/649] 10.222.0.71 () {56 vars in 1137 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:27.313040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:27.316841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:27.317735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:27.317735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:27.317735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:27.317735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:27.317968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.318041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 325/650] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:27.318041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.414962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69065201-0443-4f36-8474-d0bfd2b6e359 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.416152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611b63-11cb-4011-9b9f-d7e15032ebff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.448439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.448773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.449232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.449513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.449619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.449786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.449881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.450124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.456891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.457811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.458331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.459250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:27.472787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:27.473771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 326/651] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:27.475347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:27.476188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:27.477066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:27.478012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 326/652] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:27.480301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.577616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9095f-cd6a-4deb-bd89-018279d50318 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.577978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee19bf-5c5c-4bab-a86c-7bafed81ce2e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.610478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.611026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.611159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.611412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.611705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.611803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.611990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.612269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.612440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:27.612726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.614081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.625296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 327/653] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:27.626486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.678393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.679786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.721965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe56f6-7d85-49f1-b20a-4c6e5af296aa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.753665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.754314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.754591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.754867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.755664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:27.755965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.757366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.768672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 328/654] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:27.769441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.776724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 327/655] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:27.776724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.862114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db9800-3f3e-48f1-85b2-aaf4622ee73e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.870763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9fbb3-d93a-4967-8160-34ec3b6c5b56 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:27.893941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d821ab6-c1a7-44e3-a3b9-999d01964d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.894605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821ab6-c1a7-44e3-a3b9-999d01964d13 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.894882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821ab6-c1a7-44e3-a3b9-999d01964d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.895159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821ab6-c1a7-44e3-a3b9-999d01964d13 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.896011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d821ab6-c1a7-44e3-a3b9-999d01964d13 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:27.896318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d821ab6-c1a7-44e3-a3b9-999d01964d13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.897733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d821ab6-c1a7-44e3-a3b9-999d01964d13 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.902212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:27.902994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.903270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.903536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.911593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:27.913031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:27.915435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 329/656] 10.222.0.71 () {56 vars in 1139 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:27.916734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b86d-7f9a-4565-b28e-4b6ed23df3a6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:27.917023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b86d-7f9a-4565-b28e-4b6ed23df3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:27.917298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b86d-7f9a-4565-b28e-4b6ed23df3a6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:27.918671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 330/657] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:27.919203 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b86d-7f9a-4565-b28e-4b6ed23df3a6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:27.928168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:27.929183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 328/658] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:27.930184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.015219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b86d-7f9a-4565-b28e-4b6ed23df3a6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.026033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0214d4-aa7a-40bb-8e93-c8aca4c8c75a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.047313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.048006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.048284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.048554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.049370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=1f2421c1bfb5402da3179e6f3c749374) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:28.049698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.051120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.056641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5946c-763c-4279-b78a-a772319bbcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.056783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 331/659] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/role_assignments?user.id=1f2421c1bfb5402da3179e6f3c749374&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:28.057411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.057522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5946c-763c-4279-b78a-a772319bbcde None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.057689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5946c-763c-4279-b78a-a772319bbcde None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.057951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5946c-763c-4279-b78a-a772319bbcde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.078729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5946c-763c-4279-b78a-a772319bbcde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.080265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5946c-763c-4279-b78a-a772319bbcde None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-17a5946c-763c-4279-b78a-a772319bbcde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:28.098907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:28.100143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:28.101079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:28.101079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:28.101079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:28.101079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:28.103968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 329/660] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:28.105630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e12b10-0375-4fd1-a9cc-b47510bfbc82 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.105919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e12b10-0375-4fd1-a9cc-b47510bfbc82 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.106195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e12b10-0375-4fd1-a9cc-b47510bfbc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.124425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-65e12b10-0375-4fd1-a9cc-b47510bfbc82 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:28.125866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-65e12b10-0375-4fd1-a9cc-b47510bfbc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:28.166081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f1075-2912-4e56-8d5e-7396232795a5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.199754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 330/661] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:28.199754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e12b10-0375-4fd1-a9cc-b47510bfbc82 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.201706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.202372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.202649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.202917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.272821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.274288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.310402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e12b10-0375-4fd1-a9cc-b47510bfbc82 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.345199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.346131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.346565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.346977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.348090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:28.348550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.349973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.367233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 331/662] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:28.367851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.379262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 332/663] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:28.380312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.467214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7774432-4b68-415e-83d4-3f9fdf8379d7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.475188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecdb2-c730-4c14-9254-dc4b4071b83d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.502144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.503204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.503501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.503802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.504526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:28.504837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.506226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.509550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.510556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.511020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.511465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.517065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 332/664] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:28.517862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.526192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:28.534339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:28.535298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:28.536358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:28.542467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 333/665] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:28.543530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.618180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527e77e-2fc4-4409-9c93-a70d684fd543 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.648397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361408d1-82dc-4022-8d90-9e36a2a5bbb2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.650209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.650964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.651245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.651515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.682961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.683770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.684057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.684333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.695293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.696785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:28.713964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:28.714887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:28.721379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:28.721379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:28.721379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:28.721379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:28.721379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 334/666] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:28.721379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.721754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.721754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.816883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4a3ee-cba4-4742-8343-997d140f3d87 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.823198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 333/667] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:28.823198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.851022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.851856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.852145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.852418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.859947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.861408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:28.875608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:28.876735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 335/668] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:28.877726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:28.931321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344dd669-ff10-49b5-ae26-6f17562a9f5c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.964170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:28.964946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:28.965151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:28.965377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:28.966124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:28.966691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:28.968325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:28.976098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86404323-24dc-41a6-b5a4-d3e7fca8b8cc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:28.980637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 334/669] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:28.981294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.010344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.011009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.011291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.011567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.012316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:29.012622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.014807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.025826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 336/670] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:29.026611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.084963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144744b-a0ed-445e-a5d9-77fa7e2547b5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.116491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.117265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.117542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.117809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.122743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68756c7e-aff3-485a-8aef-9b886cdc9e76 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.128144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.129576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:29.146652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:29.147562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:29.148494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:29.148494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:29.148494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:29.148494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:29.148494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 335/671] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:29.148769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.164074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.165074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.165700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.166170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.167037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:29.167349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.168985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.180861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 337/672] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:29.181679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.253702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ecac0-ce3e-4c5e-bd39-91e80be4cfe2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.281569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24b18-ed64-4192-8169-2ae298f69c02 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.289954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.291018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.291516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.292016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.303524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.305006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.315662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.316704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.317144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.321150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:29.322207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:29.323073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:29.324372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:29.324372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:29.324372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:29.324372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:29.324372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 336/673] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:29.324372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.329034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.330495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:29.345447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:29.346429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 338/674] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:29.352329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.425532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe1b5e-8826-4732-b6d5-8c371aabfa12 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.450943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ab41-779e-4b89-a5a3-48a9e18cc901 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.457950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.458652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.458886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.459157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.459933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:29.460214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.461568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.478424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 337/675] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:29.478424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.484334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.485100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.485379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.485654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.486477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=b60d4a75e5b8464a8c7935e7433cf170) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:29.486791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.488310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.494483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 339/676] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/role_assignments?user.id=b60d4a75e5b8464a8c7935e7433cf170&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:29.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.577741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f04311-4521-4437-b5e0-8bd87c8b019c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.613879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178a325-99ab-41fe-8712-bf73b0cf56b0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.615761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.616437 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.616710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.616977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.618550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:29.618550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.619366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.632162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 338/677] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:29.633160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.652753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.653806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.654315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.655608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.656665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:29.657184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.658894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.673143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 340/678] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:29.674338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.743347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a09487-b78d-49d6-b3b6-d318205dbdac None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.778504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.779718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.780259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.780716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.789393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.790874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.793386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33086fc-2d98-4d98-9d84-01ef42c89519 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:29.805479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:29.806433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:29.807362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 339/679] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:29.807906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.835808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.836530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.836807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.837166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.850627 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.852333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:29.873313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:29.874800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:29.875885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:29.878133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 341/680] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:29.879093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:29.915464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22deee-4aaa-46de-b0e4-61595c757644 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:29.951053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:29.952456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:29.952881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:29.953294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:29.954339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=76b2a2d381a14f5fb04e721905b36ace) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:29.954804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:29.956330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:29.967654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 340/681] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/role_assignments?user.id=76b2a2d381a14f5fb04e721905b36ace&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:29.967654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:30.005057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab67628-1f30-425c-af06-b8a690ae9c12 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:30.078416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:30.079096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.079376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.079670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.080472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:30.080775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:30.083941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:30.102453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 342/682] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:30.110791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:30.144690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba89cc-a360-4674-95bc-da0d8a72a94f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:30.215135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:30.216089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.217076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.217884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.219056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:30.219363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:30.221606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:30.247581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 341/683] 10.222.0.71 () {56 vars in 1135 bytes} [Mon May 16 14:56:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:30.253395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:30.273385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b31d6-87ea-47c6-89ee-effaf527e2b8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:30.312508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03382a6c-99e4-42e5-82ce-c809a3a3a04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:30.322808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382a6c-99e4-42e5-82ce-c809a3a3a04a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.322808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382a6c-99e4-42e5-82ce-c809a3a3a04a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.322808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382a6c-99e4-42e5-82ce-c809a3a3a04a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.322808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03382a6c-99e4-42e5-82ce-c809a3a3a04a None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=3d58a6e2eac047ad8e88a1f973ecc697) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:30.322808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03382a6c-99e4-42e5-82ce-c809a3a3a04a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:30.322808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03382a6c-99e4-42e5-82ce-c809a3a3a04a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:30.324863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 343/684] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:30 2022] GET /identity/v3/role_assignments?user.id=3d58a6e2eac047ad8e88a1f973ecc697&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:30.383113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84470a6-4b08-4760-bfb1-1b5086929843 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:30.419903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:30.421120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.421557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.421831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.438870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c0dab-beb6-4992-8e4f-9e94f9285859 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.439555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c0dab-beb6-4992-8e4f-9e94f9285859 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.439555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c0dab-beb6-4992-8e4f-9e94f9285859 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.441031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 344/685] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:30.447236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9abec-e4cf-465d-a4da-af9209bedea6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.447542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9abec-e4cf-465d-a4da-af9209bedea6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.447825 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9abec-e4cf-465d-a4da-af9209bedea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.469024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-51b9abec-e4cf-465d-a4da-af9209bedea6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:30.470362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-51b9abec-e4cf-465d-a4da-af9209bedea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:30.514319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:30.517206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:30.547406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 345/686] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:30.572593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9abec-e4cf-465d-a4da-af9209bedea6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:30.664947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 342/687] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:30.669883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:30.698478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9abec-e4cf-465d-a4da-af9209bedea6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:30.740199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:30.741026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.741300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.741567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.742221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:30.743592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:30.759038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:30.760130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:30.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 346/688] 10.222.0.71 () {56 vars in 1129 bytes} [Mon May 16 14:56:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:30.766928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:30.781037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7909d-53c8-4854-8a57-06e359306d74 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:30.816780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:30.817575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.817853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.818190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.831310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:30.833442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:30.855199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:30.857267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:30.860595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:30.860595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:30.860595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:30.860595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:30.860595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 343/689] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:30.867653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:30.879014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6cc5-1989-46b0-9a47-7c9a40631c9e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:30.915140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:30.915797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:30.916058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:30.916334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:30.917039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:30.917337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:30.918708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:30.924408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 347/690] 10.222.0.71 () {56 vars in 1124 bytes} [Mon May 16 14:56:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:30.929501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:31.029645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5618e6-00ea-4bb2-a38a-cbf9cd34eda7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:31.098146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9889d14-af73-41ed-8060-fbada7a1f709 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:31.100350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:31.101029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:31.101304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:31.101574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:31.102305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:31.102607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:31.104075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:31.128571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 344/691] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:31.133005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:31.158591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:31.159262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:31.159596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:31.159903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:31.162027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:31.162332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:31.164503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:31.176607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 348/692] 10.222.0.71 () {56 vars in 1124 bytes} [Mon May 16 14:56:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:31.292969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43c5f8-0aab-46f7-9efa-004fe30c479f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:31.342846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:31.343688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:31.343974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:31.344246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:31.353846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:31.355293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:31.376365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:31.379448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:31.380397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 345/693] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:31.386256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:31.491705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb890549-5a8a-464e-b51c-3d7d8a10bd38 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:31.526725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1128c0-68d1-4759-a458-57f28b9f3c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:31.527395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1128c0-68d1-4759-a458-57f28b9f3c7f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:31.527699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1128c0-68d1-4759-a458-57f28b9f3c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:31.527979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1128c0-68d1-4759-a458-57f28b9f3c7f None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:31.528771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1128c0-68d1-4759-a458-57f28b9f3c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:31.529080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1128c0-68d1-4759-a458-57f28b9f3c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:31.531262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1128c0-68d1-4759-a458-57f28b9f3c7f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:31.544123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 349/694] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:31.550438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:31.657817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ffa62-1b88-408c-aa4b-376f4a744f19 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:31.692408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6837f4b-7e4d-4088-83a7-a76250d6a148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:31.693419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837f4b-7e4d-4088-83a7-a76250d6a148 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:31.693922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837f4b-7e4d-4088-83a7-a76250d6a148 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:31.694411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837f4b-7e4d-4088-83a7-a76250d6a148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:31.695447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6837f4b-7e4d-4088-83a7-a76250d6a148 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=918d67038f9c4d0fa4d2456409245774) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:31.695972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6837f4b-7e4d-4088-83a7-a76250d6a148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:31.697594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6837f4b-7e4d-4088-83a7-a76250d6a148 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:31.705143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 346/695] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:31 2022] GET /identity/v3/role_assignments?user.id=918d67038f9c4d0fa4d2456409245774&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:32.220966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12740cbf-b60d-4256-bd18-d37b416bcc51 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:32.221318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12740cbf-b60d-4256-bd18-d37b416bcc51 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:32.221643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12740cbf-b60d-4256-bd18-d37b416bcc51 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:32.223272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 350/696] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:32.228679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aa9df-83d1-4f62-ae64-a3d366f0afb1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:32.229121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aa9df-83d1-4f62-ae64-a3d366f0afb1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:32.229212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aa9df-83d1-4f62-ae64-a3d366f0afb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:32.246755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-562aa9df-83d1-4f62-ae64-a3d366f0afb1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:32.247965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-562aa9df-83d1-4f62-ae64-a3d366f0afb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:32.316562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 347/697] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:32.343030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12740cbf-b60d-4256-bd18-d37b416bcc51 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:32.486046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12740cbf-b60d-4256-bd18-d37b416bcc51 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:32.537513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:32.538587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:32.539038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:32.539454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:32.557505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:32.559231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:32.582694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:32.583855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:32.584962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:32.584962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:32.584962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:32.584962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:32.591165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 351/698] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:32.592988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aa9df-83d1-4f62-ae64-a3d366f0afb1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:32.689292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aa9df-83d1-4f62-ae64-a3d366f0afb1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:32.736497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219f950-6a02-46b3-b958-f9b2798a785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:32.737457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219f950-6a02-46b3-b958-f9b2798a785b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:32.737911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219f950-6a02-46b3-b958-f9b2798a785b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:32.738417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219f950-6a02-46b3-b958-f9b2798a785b None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:32.739319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219f950-6a02-46b3-b958-f9b2798a785b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:32.739836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219f950-6a02-46b3-b958-f9b2798a785b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:32.741378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219f950-6a02-46b3-b958-f9b2798a785b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:32.754396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 348/699] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:32.777853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:32.827268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f7167-c07a-4c52-9a72-fb7e8f202e93 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:32.827565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f7167-c07a-4c52-9a72-fb7e8f202e93 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:32.827853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f7167-c07a-4c52-9a72-fb7e8f202e93 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:32.829572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 349/700] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:32.834901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad4fb8-f472-4612-8830-7accec6ea8a8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:32.835181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad4fb8-f472-4612-8830-7accec6ea8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:32.835452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad4fb8-f472-4612-8830-7accec6ea8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:32.864500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-01ad4fb8-f472-4612-8830-7accec6ea8a8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:32.865687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-01ad4fb8-f472-4612-8830-7accec6ea8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:32.949258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce92c6-e86d-44c1-95a6-05d0db31bf06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:32.994077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 350/701] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 16 14:56:33.007179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:33.008174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.008655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.009121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.010032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:33.011695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:33.039436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad4fb8-f472-4612-8830-7accec6ea8a8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:33.108979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 352/702] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:56:32 2022] POST /identity/v3/users => generated 254 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:33.153210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad4fb8-f472-4612-8830-7accec6ea8a8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:33.190287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c15e49-a66b-48b3-bc32-564919cc60e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:33.191884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c15e49-a66b-48b3-bc32-564919cc60e7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.192344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c15e49-a66b-48b3-bc32-564919cc60e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.192791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c15e49-a66b-48b3-bc32-564919cc60e7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.193641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c15e49-a66b-48b3-bc32-564919cc60e7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:33.195220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c15e49-a66b-48b3-bc32-564919cc60e7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-01c15e49-a66b-48b3-bc32-564919cc60e7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:33.210789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:33.211891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:33.212726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:33.215863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 351/703] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:33 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:33.223168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:33.228555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae12a58-c696-45e3-9b76-b3b3f307475c None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.228847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae12a58-c696-45e3-9b76-b3b3f307475c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.229392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae12a58-c696-45e3-9b76-b3b3f307475c None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.230733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 352/704] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:33.237073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b818d0f-9a46-42eb-8d9b-ab08e1278cb1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.237397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b818d0f-9a46-42eb-8d9b-ab08e1278cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.237624 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b818d0f-9a46-42eb-8d9b-ab08e1278cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.268300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2b818d0f-9a46-42eb-8d9b-ab08e1278cb1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:33.269258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2b818d0f-9a46-42eb-8d9b-ab08e1278cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:33.350188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd5600-93c0-4a23-930d-212f9b54ac77 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:33.387681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:33.388820 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.389935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.390775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.392042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:33.392545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:33.394237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:33.396500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 353/705] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:33.399253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 353/706] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:56:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:33.404654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b818d0f-9a46-42eb-8d9b-ab08e1278cb1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:33.421749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:33.519243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b818d0f-9a46-42eb-8d9b-ab08e1278cb1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:33.554949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9e279-df33-4f44-a92f-77ed2640e719 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:33.556759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:33.558900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.558900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.558900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.558900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] RBAC: Adding query filter params (type=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:33.559210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:33.560318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:33.566835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 354/707] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:56:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:33.600149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:33.601342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.601848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.602317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.616049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:33.624424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:33.625573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:33.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:33.629392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 354/708] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:33.634878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:33.746546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0dff7-2bd7-4aae-8c8f-e6d4b4e91c7b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:33.783819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:33.784817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.785311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.786043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.787547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:33.788089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:33.789703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:33.800415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 355/709] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:33.806043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:33.932285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c8ad-0d31-42c0-9265-fdf766b35dbd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:33.971071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165513d-a904-48ba-a9c6-98e8257c6dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:33.972274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165513d-a904-48ba-a9c6-98e8257c6dad None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:33.972755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165513d-a904-48ba-a9c6-98e8257c6dad None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:33.973224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165513d-a904-48ba-a9c6-98e8257c6dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:33.984217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165513d-a904-48ba-a9c6-98e8257c6dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:33.986070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165513d-a904-48ba-a9c6-98e8257c6dad None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1165513d-a904-48ba-a9c6-98e8257c6dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:34.004389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:34.005356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:34.006251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:34.006251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:34.006251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:34.006251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:34.008652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 355/710] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:34.013736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:34.073558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972f1c-4d98-4cbd-a424-f1fdc5bb250f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:34.073742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972f1c-4d98-4cbd-a424-f1fdc5bb250f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:34.074014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972f1c-4d98-4cbd-a424-f1fdc5bb250f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:34.075922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 356/711] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:34.089258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76665-f807-4716-894b-07a069ffa5a8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:34.089546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76665-f807-4716-894b-07a069ffa5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:34.089817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76665-f807-4716-894b-07a069ffa5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:34.120094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-20a76665-f807-4716-894b-07a069ffa5a8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:34.121299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-20a76665-f807-4716-894b-07a069ffa5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:34.128897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb2508-778d-478f-b8d9-d2f22297af5e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:34.164002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:34.164991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:34.165272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:34.165544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:34.166264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:34.166567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:34.168138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:34.181717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 356/712] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:34.188740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:34.239723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 357/713] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:34.274301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76665-f807-4716-894b-07a069ffa5a8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:34.371280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452145cb-6fda-47c2-9771-c57f1a5ecd4d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:34.385010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76665-f807-4716-894b-07a069ffa5a8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:34.425077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:34.426007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:34.426290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:34.426563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:34.427288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:34.428849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:34.443234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:34.445061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:34.445061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:34.445061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:34.453351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:34.455039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:34.470445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:34.472791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:34.475012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:34.479511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 357/714] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:56:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:34.479511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:34.532249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 358/715] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:56:34 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 14:56:34.595706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a351-9ce3-4bbf-b1c8-10fb0a3f030f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:34.645129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:34.645719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:34.645994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:34.646263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:34.647099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:34.647401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:34.648849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:34.668554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 358/716] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:56:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:34.674362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:34.816714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75270c-b598-4077-931e-995ab00f9c6d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:34.863473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad97105-41b3-47a7-9f48-97c28dfa6d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:34.864181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97105-41b3-47a7-9f48-97c28dfa6d02 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:34.864458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97105-41b3-47a7-9f48-97c28dfa6d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:34.864728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97105-41b3-47a7-9f48-97c28dfa6d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:34.955123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad97105-41b3-47a7-9f48-97c28dfa6d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:34.956954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad97105-41b3-47a7-9f48-97c28dfa6d02 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:35.081670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 359/717] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:34 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 14:56:35.087576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:35.109372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c7db7-3daf-4665-811a-310c3332c5f8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.109626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c7db7-3daf-4665-811a-310c3332c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.109903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c7db7-3daf-4665-811a-310c3332c5f8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.111516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 360/718] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:35.116451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c3654-2c09-4f65-a9c9-19e5ea65e950 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.116737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c3654-2c09-4f65-a9c9-19e5ea65e950 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.117011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c3654-2c09-4f65-a9c9-19e5ea65e950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.134489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f02c3654-2c09-4f65-a9c9-19e5ea65e950 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:35.135857 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f02c3654-2c09-4f65-a9c9-19e5ea65e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:35.218602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 361/719] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 14:56:35.256843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c3654-2c09-4f65-a9c9-19e5ea65e950 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:35.297075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a5cf3-6947-4659-b3cc-a9af26915c59 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:35.365263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:35.366066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.366351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.366627 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.388481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:35.390057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:35.394851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c3654-2c09-4f65-a9c9-19e5ea65e950 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:35.419537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:35.421296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:35.422504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:35.422504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:35.422504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:35.422504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:35.422504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 359/720] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:35.426876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:35.434545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:35.435723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.436163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.437651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.462470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:35.473196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:35.474172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:35.475037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:35.477872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 362/721] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:35.485741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:35.607414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7edfa-8f24-4a2b-b43a-5304f571fe6e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:35.670814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01205b38-c22a-4f4d-8cd3-314764ec1fb6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:35.679657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:35.680328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.680607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.680880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.681614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:35.681917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:35.683293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:35.707724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 360/722] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:35.712216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:35.745253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:35.745921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.746129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.746405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.747160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:35.747466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:35.750615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:35.765409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 363/723] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:35.776609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:35.846948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49496ce5-02a9-414f-8f3c-7f5b5fdcf051 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:35.890656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:35.891467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.891767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.892037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.899127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:35.900600 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:35.918285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515108bc-b023-4978-8804-97948bf77ac7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:35.929758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:35.930716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:35.931613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 361/724] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:35.938877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:35.964352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:35.965158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:35.965436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:35.965709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:35.986510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:35.988011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:36.017220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:36.019249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:36.021427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:36.021427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:36.021427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:36.021427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:36.027825 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 364/725] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:36.036849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:36.052671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88864f77-d26b-4d87-b87b-55dcaa6ede95 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:36.101290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:36.101961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.102240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.102512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.103304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:36.103894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:36.105440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:36.132099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 362/726] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:36.137654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:36.176154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cc28e-b654-4e94-9c85-dc48b3fdca11 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:36.253504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec454c6-69ea-4954-993d-d4f86166ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:36.254197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec454c6-69ea-4954-993d-d4f86166ea27 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.254479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec454c6-69ea-4954-993d-d4f86166ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.254755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec454c6-69ea-4954-993d-d4f86166ea27 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.255510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec454c6-69ea-4954-993d-d4f86166ea27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:36.255835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec454c6-69ea-4954-993d-d4f86166ea27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:36.257263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec454c6-69ea-4954-993d-d4f86166ea27 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:36.271252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 365/727] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:36.272568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbefe66-023f-4751-b29c-54a1efd400b5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.272858 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbefe66-023f-4751-b29c-54a1efd400b5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.273129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbefe66-023f-4751-b29c-54a1efd400b5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.274616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 366/728] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:36.280219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9f3e-4531-45d9-84bb-b2c218ccc979 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.280528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9f3e-4531-45d9-84bb-b2c218ccc979 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.280771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9f3e-4531-45d9-84bb-b2c218ccc979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.293453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faff7db-471c-4afb-9e92-d362a856c892 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:36.297357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2cad9f3e-4531-45d9-84bb-b2c218ccc979 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:36.298526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2cad9f3e-4531-45d9-84bb-b2c218ccc979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:36.329241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:36.330294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.330893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.331374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.332545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=3d58a6e2eac047ad8e88a1f973ecc697) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:36.333115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:36.334750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:36.341030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 363/729] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:36 2022] GET /identity/v3/role_assignments?user.id=3d58a6e2eac047ad8e88a1f973ecc697&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:36.342427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:36.376058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 367/730] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 14:56:36.398899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9f3e-4531-45d9-84bb-b2c218ccc979 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:36.449428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054accae-72f2-46fe-a6f6-df6b5f0d23d0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:36.488212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:36.489388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.489898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.490415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.499206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:36.502334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:36.518244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:36.519297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:36.520320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:36.527457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 364/731] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:36.529160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:36.634973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f9bf-8bec-424e-a56c-f4e724ec4cae None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:36.641451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9f3e-4531-45d9-84bb-b2c218ccc979 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:36.670102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043e386-f06c-4533-9d0d-ace99482563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:36.671186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e386-f06c-4533-9d0d-ace99482563a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.671729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e386-f06c-4533-9d0d-ace99482563a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.672220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e386-f06c-4533-9d0d-ace99482563a None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.673263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043e386-f06c-4533-9d0d-ace99482563a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:36.673778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043e386-f06c-4533-9d0d-ace99482563a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:36.675452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043e386-f06c-4533-9d0d-ace99482563a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:36.697701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 365/732] 10.222.0.71 () {56 vars in 1135 bytes} [Mon May 16 14:56:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:36.697701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d68bd-b8ff-468c-84ed-e50dfabfb172 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.697701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d68bd-b8ff-468c-84ed-e50dfabfb172 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.697701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d68bd-b8ff-468c-84ed-e50dfabfb172 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.698926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 366/733] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:36.699616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d68bd-b8ff-468c-84ed-e50dfabfb172 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:36.711484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99ca0f-1e05-476a-8a67-74918fe9957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:36.712818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ca0f-1e05-476a-8a67-74918fe9957e None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.713576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ca0f-1e05-476a-8a67-74918fe9957e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.714081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ca0f-1e05-476a-8a67-74918fe9957e None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.715210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99ca0f-1e05-476a-8a67-74918fe9957e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:36.718295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99ca0f-1e05-476a-8a67-74918fe9957e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:36.798210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 368/734] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:56:36 2022] POST /identity/v3/services => generated 240 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 14:56:36.798210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2088339-2873-478d-aa02-0ef882178d04 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.798210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2088339-2873-478d-aa02-0ef882178d04 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.798938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2088339-2873-478d-aa02-0ef882178d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.834653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b2088339-2873-478d-aa02-0ef882178d04 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:36.835895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b2088339-2873-478d-aa02-0ef882178d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:36.853971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d68bd-b8ff-468c-84ed-e50dfabfb172 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:36.927908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:36.928630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:36.928914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:36.929188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:36.994147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 369/735] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:37.017650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2088339-2873-478d-aa02-0ef882178d04 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.043198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.045047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:37.118130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2088339-2873-478d-aa02-0ef882178d04 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.152683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 367/736] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:56:37.155501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.159061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.160199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.160684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.161173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.175535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:37.185453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:37.186508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:37.187901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:37.189433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 370/737] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:37.197167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.266829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf9772-41d0-4fa9-af08-fb65eb2c6d2c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.302583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.303781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.304322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.304801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.306372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c3201-67a8-4fe0-aa5f-4bdfd9148621 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.317158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.319023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:37.340102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:37.341148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:37.343127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:37.343127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:37.343127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:37.343127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:37.349683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 368/738] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:37.351549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.352271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.352548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.352817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.353440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.354853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:37.355156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.356594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:37.366512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 371/739] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:37.380633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.456807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922f41b-4bdc-4711-b26a-f10eac7d609b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.491930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.492966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.493446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.493965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.494893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:37.495392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.497006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:37.515161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 369/740] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:37.516746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.573822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054482ba-d317-4ec9-baed-4e84e3b45fff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.619807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee04c3-6210-4fd7-af0d-a697cb5de0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.620609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee04c3-6210-4fd7-af0d-a697cb5de0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.620953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee04c3-6210-4fd7-af0d-a697cb5de0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.621162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee04c3-6210-4fd7-af0d-a697cb5de0f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.629837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e443902-7803-4394-8fd6-116fb52745cf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.632183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee04c3-6210-4fd7-af0d-a697cb5de0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.633785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee04c3-6210-4fd7-af0d-a697cb5de0f7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0bee04c3-6210-4fd7-af0d-a697cb5de0f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:37.650905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:37.651915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:37.653287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:37.653287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:37.653287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:37.653287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:37.653287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 372/741] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:37.654366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d540b9-0604-4113-a472-c05a1b424020 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.654728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d540b9-0604-4113-a472-c05a1b424020 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.655076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d540b9-0604-4113-a472-c05a1b424020 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.661746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 373/742] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:37.662546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d540b9-0604-4113-a472-c05a1b424020 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.676511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8ac97-0e0c-46e5-858d-120a3ed873f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.677893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8ac97-0e0c-46e5-858d-120a3ed873f4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.678353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8ac97-0e0c-46e5-858d-120a3ed873f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.678953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8ac97-0e0c-46e5-858d-120a3ed873f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.688455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8ac97-0e0c-46e5-858d-120a3ed873f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.691648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8ac97-0e0c-46e5-858d-120a3ed873f4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-38e8ac97-0e0c-46e5-858d-120a3ed873f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:37.710123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:37.711228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:37.712356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:37.720105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 370/743] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:37.720105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652cc0-b058-408e-b819-68fc60b1d488 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.720105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652cc0-b058-408e-b819-68fc60b1d488 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.720105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652cc0-b058-408e-b819-68fc60b1d488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.738228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c8652cc0-b058-408e-b819-68fc60b1d488 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:37.739416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c8652cc0-b058-408e-b819-68fc60b1d488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:37.790391 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d540b9-0604-4113-a472-c05a1b424020 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.809772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 371/744] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:37.810638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652cc0-b058-408e-b819-68fc60b1d488 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.846978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.846978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.846978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.846978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.847734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:37.847734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.848209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:37.861421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 374/745] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:37.862292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:37.945822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652cc0-b058-408e-b819-68fc60b1d488 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.962999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dd87e-a046-4665-8e76-b05b14d12cac None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:37.988277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:37.988946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:37.989232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:37.989543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:37.990293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:37.990735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:37.992514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.001296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.002116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.002396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.002669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.003321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.004755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.006224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 372/746] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:38.007484 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:38.017992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:38.018961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:38.020222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 375/747] 10.222.0.71 () {56 vars in 1129 bytes} [Mon May 16 14:56:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:38.022226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.128830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bb153-4845-4f2c-8c66-50bfff616ece None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:38.149098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e572a-8310-4839-86b7-4d392378374f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:38.161365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.162181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.162458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.162725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.170092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.172276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.187899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:38.188345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:38.190615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:38.192898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:38.199092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.199092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.199092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.199610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 373/748] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:38.199782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=918d67038f9c4d0fa4d2456409245774) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:38.199782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.200067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.200210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.200210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 376/749] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/role_assignments?user.id=918d67038f9c4d0fa4d2456409245774&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:38.204673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.339560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54d1b6-9b6e-428e-af91-79355b959d73 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:38.389348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.390614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.390882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.391270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.392069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:38.392311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.393741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.407119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 374/750] 10.222.0.71 () {56 vars in 1124 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:38.412786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.449297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea85ca7-bde9-44ec-8542-00f474317bf2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:38.520344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.521033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.521316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.521593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.522445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:38.522753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.524212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.553176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a508a0f-4acc-4f9b-9786-dc714c3448c7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:38.565860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 377/751] 10.222.0.71 () {56 vars in 1149 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:38.571279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.599406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.600428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.600907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.601393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.602297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:38.603909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.605497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.621063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 375/752] 10.222.0.71 () {56 vars in 1124 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:38.623973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.712289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f867e8-5bd5-41ff-9fc2-a4ada19f1171 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:38.755180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.757306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.757593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.757866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.833483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.834949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.893896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328fc59-9c58-43a3-ac61-cf720a4fe910 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:38.935107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59836e-a34b-4b86-af1f-347bf8d7fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:38.935818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59836e-a34b-4b86-af1f-347bf8d7fe26 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.936096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59836e-a34b-4b86-af1f-347bf8d7fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.936364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59836e-a34b-4b86-af1f-347bf8d7fe26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.937222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59836e-a34b-4b86-af1f-347bf8d7fe26 None admin] RBAC: Adding query filter params (interface=public, service_id=b9502db1433c401d8a86f9a0a05fdb71, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:38.937622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59836e-a34b-4b86-af1f-347bf8d7fe26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:38.939004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59836e-a34b-4b86-af1f-347bf8d7fe26 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:38.947057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 376/753] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/endpoints?service_id=b9502db1433c401d8a86f9a0a05fdb71&interface=public®ion_id=RegionOne => generated 194 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:38.950307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 378/754] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 14:56:38.954984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3263d0-829a-4894-9be5-003e7208fe1a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:38.954984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3263d0-829a-4894-9be5-003e7208fe1a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:38.954984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3263d0-829a-4894-9be5-003e7208fe1a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:38.955557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 377/755] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:38.956302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3263d0-829a-4894-9be5-003e7208fe1a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:38.956563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:39.068704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3263d0-829a-4894-9be5-003e7208fe1a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.086881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0644e8-1e18-40cf-8787-b07acbf287c1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.104290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:39.105095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.105377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.105651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.115933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:39.117619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:39.128254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fb827-3d8b-4bf2-9739-b2be790870ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:39.129406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb827-3d8b-4bf2-9739-b2be790870ec None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.129826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb827-3d8b-4bf2-9739-b2be790870ec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.130239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb827-3d8b-4bf2-9739-b2be790870ec None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.130926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fb827-3d8b-4bf2-9739-b2be790870ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:39.132339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fb827-3d8b-4bf2-9739-b2be790870ec None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:39.138058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 379/756] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:39.139344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b9efd-e4fd-4bd8-b94c-8da7601014d6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.139650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b9efd-e4fd-4bd8-b94c-8da7601014d6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.140752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b9efd-e4fd-4bd8-b94c-8da7601014d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:39.144051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:39.144993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:39.147283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:39.147283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:39.147283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:39.147283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:39.147283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 378/757] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:39.161088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:39.177760 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-061b9efd-e4fd-4bd8-b94c-8da7601014d6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:39.178994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-061b9efd-e4fd-4bd8-b94c-8da7601014d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:39.276881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 380/758] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:39.299157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b9efd-e4fd-4bd8-b94c-8da7601014d6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:39.386194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61496dc9-708e-4e50-845a-63d261288502 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.440906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:39.440906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.440906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.440906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.442082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:39.442082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:39.443321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:39.462094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 379/759] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:39.468687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:39.564910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b9efd-e4fd-4bd8-b94c-8da7601014d6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.588757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc685b2-0d25-473b-9908-e284c67f6e87 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.602319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6c00e-e8b3-4767-8273-20a9319f6069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:39.603120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6c00e-e8b3-4767-8273-20a9319f6069 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.603400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6c00e-e8b3-4767-8273-20a9319f6069 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.603696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6c00e-e8b3-4767-8273-20a9319f6069 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.614214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6c00e-e8b3-4767-8273-20a9319f6069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3cd6c00e-e8b3-4767-8273-20a9319f6069 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:39.621859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:39.622830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:39.623704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 381/760] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:39.625546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021735e9-081c-4cb0-aa62-042f47b368e0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.625546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021735e9-081c-4cb0-aa62-042f47b368e0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.625546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021735e9-081c-4cb0-aa62-042f47b368e0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.626874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 382/761] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:39.630953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021735e9-081c-4cb0-aa62-042f47b368e0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:39.638744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e7ce0-ad24-4e86-a60c-0af8b0b00482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:39.640300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e7ce0-ad24-4e86-a60c-0af8b0b00482 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.641145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e7ce0-ad24-4e86-a60c-0af8b0b00482 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.641634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e7ce0-ad24-4e86-a60c-0af8b0b00482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.651026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e7ce0-ad24-4e86-a60c-0af8b0b00482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:39.652739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e7ce0-ad24-4e86-a60c-0af8b0b00482 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d40e7ce0-ad24-4e86-a60c-0af8b0b00482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:39.670100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:39.671212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:39.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:39.674215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 380/762] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:39.680092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607722f-7947-4c26-9b56-2ef28476d986 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.680549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607722f-7947-4c26-9b56-2ef28476d986 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.680933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607722f-7947-4c26-9b56-2ef28476d986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.698683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a607722f-7947-4c26-9b56-2ef28476d986 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:39.699997 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a607722f-7947-4c26-9b56-2ef28476d986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:39.753183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021735e9-081c-4cb0-aa62-042f47b368e0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.775657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 381/763] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:39.775657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607722f-7947-4c26-9b56-2ef28476d986 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:39.813117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:39.814400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.815061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.815854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.817137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:39.817665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:39.820100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:39.832092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 383/764] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:39.832944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:39.902207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607722f-7947-4c26-9b56-2ef28476d986 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.946467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:39.947489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:39.948007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:39.948482 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:39.950135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:39.950664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:39.953490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:39.960912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d20d0-ef3f-453f-ada3-1ce7d6939842 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:39.972389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 382/765] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:39.973576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.001892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.003521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.004297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.005071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.006442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.008508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:40.024201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:40.025239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:40.027168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 384/766] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:39 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:40.028023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.095369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560e742-2b0c-4290-a8d6-cd6977dbc05e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.127707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26614-97d0-4f03-aee4-337b6ff6b1a7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.130473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608975ef-b899-445e-bb65-185752bd758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.131607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608975ef-b899-445e-bb65-185752bd758b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.132200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608975ef-b899-445e-bb65-185752bd758b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.132715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608975ef-b899-445e-bb65-185752bd758b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.145254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608975ef-b899-445e-bb65-185752bd758b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.147108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608975ef-b899-445e-bb65-185752bd758b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.162410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.162997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.163273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.163543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.164405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=4dd72a14691c48dd95845b7b46eb8c58) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:40.164712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.166143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-608975ef-b899-445e-bb65-185752bd758b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:40.166785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:40.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:40.168570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:40.168570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:40.168570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:40.168570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:40.177038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 383/768] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:40.178364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608975ef-b899-445e-bb65-185752bd758b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.179294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 385/768] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/role_assignments?user.id=4dd72a14691c48dd95845b7b46eb8c58&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:40.180396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.288451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608975ef-b899-445e-bb65-185752bd758b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.328911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.329990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.330522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.330962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.332152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:40.333195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.334867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.341353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 384/769] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:40.350208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.365909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ad5a4-061f-45cf-bfe0-d12d2b4b36cd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.440961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.441570 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.441851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.442123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.442880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:40.443188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.444633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.459560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 386/770] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:40.465986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.540436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4680c-1bde-4fa1-a718-a5d8cefa25e5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.585533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.586255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.586539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.586813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.587522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] RBAC: Adding query filter params (type=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:40.587894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.589263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.600910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af4081-edce-44ad-8556-2f9a3317095e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.608118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 385/771] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:40.614460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.643425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.644273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.644549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.644816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.663842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.665293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:40.687984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:40.688989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:40.689941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:40.692072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 387/772] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:40.704628 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.746746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c7c2a-07eb-4160-a345-677f61d75e8d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.789031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743fe16-3b4e-4ee5-9026-6223080ca2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.790111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743fe16-3b4e-4ee5-9026-6223080ca2ba None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.790397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743fe16-3b4e-4ee5-9026-6223080ca2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.790676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743fe16-3b4e-4ee5-9026-6223080ca2ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.791854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743fe16-3b4e-4ee5-9026-6223080ca2ba None admin] RBAC: Adding query filter params (interface=public, service_id=2a1ce82ee27c47e497293ede9710c09d, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:40.792173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743fe16-3b4e-4ee5-9026-6223080ca2ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.794134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743fe16-3b4e-4ee5-9026-6223080ca2ba None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.801507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 386/773] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/endpoints?service_id=2a1ce82ee27c47e497293ede9710c09d&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:40.803286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e554e-e9d9-4fd8-8b9d-a9d0d335d401 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.803585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e554e-e9d9-4fd8-8b9d-a9d0d335d401 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.803889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e554e-e9d9-4fd8-8b9d-a9d0d335d401 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.805774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 387/774] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:40.806492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e554e-e9d9-4fd8-8b9d-a9d0d335d401 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:40.819670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93131a63-091a-49c2-a56b-f3af4559d9b9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.857918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e51d5f-63c3-48ee-8fbf-6b7cc52976f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.858845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e51d5f-63c3-48ee-8fbf-6b7cc52976f7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.859390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e51d5f-63c3-48ee-8fbf-6b7cc52976f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.859928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e51d5f-63c3-48ee-8fbf-6b7cc52976f7 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.860860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e51d5f-63c3-48ee-8fbf-6b7cc52976f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:40.861281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e51d5f-63c3-48ee-8fbf-6b7cc52976f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.862844 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e51d5f-63c3-48ee-8fbf-6b7cc52976f7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.887222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 388/775] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:40.888758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d139d4-b9cf-483b-b760-25b1b78d160f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.889116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d139d4-b9cf-483b-b760-25b1b78d160f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.889458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d139d4-b9cf-483b-b760-25b1b78d160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.911282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e0d139d4-b9cf-483b-b760-25b1b78d160f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:40.912505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0d139d4-b9cf-483b-b760-25b1b78d160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:40.928965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e554e-e9d9-4fd8-8b9d-a9d0d335d401 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:40.963990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:40.964664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:40.964947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:40.965220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:40.965908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:40.967271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:40.979788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 388/776] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:40.979788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.021511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 389/777] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:41.060111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d139d4-b9cf-483b-b760-25b1b78d160f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.086960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0f94-8fac-4e69-a625-9729ca4af7e4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.123277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.124117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.125194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.125592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.136816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:41.137935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:41.167751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:41.168726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:41.169612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:41.169612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:41.169612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:41.169612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:41.169612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 389/778] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:41.179688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.248712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d139d4-b9cf-483b-b760-25b1b78d160f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.288977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74869468-a360-40b2-8359-ae1700de9538 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.289492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.290519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.290996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.291461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.302680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:41.311356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:41.312906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:41.313743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:41.316363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 390/779] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:41.324558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.331262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.332223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.332685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.333129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.334022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:41.334539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:41.336129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:41.350486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 390/780] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:41.358000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.488337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69244f-7d69-40a2-a2c4-f8840d7e85bf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.537548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93befc6-ed5c-4df6-b291-51253e8dce93 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.557861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.558535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.558810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.559082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.559854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:41.560157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:41.561508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:41.579029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 391/781] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:41.585757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.620933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.621742 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.622028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.622301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.640911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:41.642368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:41.676311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:41.677293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:41.678230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 391/782] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:41.683189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.710256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c3f0a-18a7-4e67-ae25-05f429e6c950 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.753513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fb689-1699-4f9f-8e2f-508b4f2d0ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.754848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fb689-1699-4f9f-8e2f-508b4f2d0ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.755613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fb689-1699-4f9f-8e2f-508b4f2d0ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.756091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fb689-1699-4f9f-8e2f-508b4f2d0ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.770454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fb689-1699-4f9f-8e2f-508b4f2d0ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:41.772904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fb689-1699-4f9f-8e2f-508b4f2d0ae1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2d3fb689-1699-4f9f-8e2f-508b4f2d0ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:41.794120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:41.795054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:41.796691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:41.796691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:41.796691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:41.796691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:41.800365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 392/783] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:41.801707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58195863-fd62-4b38-9d44-7d13bf61fd9c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.801990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58195863-fd62-4b38-9d44-7d13bf61fd9c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.802260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58195863-fd62-4b38-9d44-7d13bf61fd9c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.803790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 393/784] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:41.804483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58195863-fd62-4b38-9d44-7d13bf61fd9c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.815051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659b724-df22-4f28-9edb-4e9949dd7df5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.863751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b0172-8efc-401d-bf14-47d45b383ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.864739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b0172-8efc-401d-bf14-47d45b383ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.865615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b0172-8efc-401d-bf14-47d45b383ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.866111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b0172-8efc-401d-bf14-47d45b383ef5 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.867384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b0172-8efc-401d-bf14-47d45b383ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:41.871614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b0172-8efc-401d-bf14-47d45b383ef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:41.873328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b0172-8efc-401d-bf14-47d45b383ef5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:41.894655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 392/785] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:41.894655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408eaa-8e64-4cfd-81dd-948f5062dde7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.894655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408eaa-8e64-4cfd-81dd-948f5062dde7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.894655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408eaa-8e64-4cfd-81dd-948f5062dde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.915187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ec408eaa-8e64-4cfd-81dd-948f5062dde7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:41.916393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ec408eaa-8e64-4cfd-81dd-948f5062dde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:41.934321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58195863-fd62-4b38-9d44-7d13bf61fd9c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:41.983025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:41.984799 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:41.985443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:41.985921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:41.986843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:41.987351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:41.987882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 393/786] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:41.988722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec408eaa-8e64-4cfd-81dd-948f5062dde7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:41.989410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.004047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 394/787] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:42.010011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.107322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec408eaa-8e64-4cfd-81dd-948f5062dde7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.112746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df48e-5000-45ed-918d-0fdd7fab5e98 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.141770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.143011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.143591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.144156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/3d58a6e2eac047ad8e88a1f973ecc697/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.146659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=3d58a6e2eac047ad8e88a1f973ecc697, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.146770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.147614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.147983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.148324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.149239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.156806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.158335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:42.173631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:42.174654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 395/788] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:42.180229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.220225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 394/789] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:41 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/3d58a6e2eac047ad8e88a1f973ecc697/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:56:42.221975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.285768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3948d8-8fc2-4f46-b065-1d05caae2928 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.322146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.322979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.323252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.323519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.324278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.325706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:42.339381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:42.340441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:42.341742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 396/790] 10.222.0.71 () {56 vars in 1129 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:42.347677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.361481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e0130-66ad-4a97-9bb4-840086476a91 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.402864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.403576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.403974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.404169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.405052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:42.406122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.407348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.426317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 395/791] 10.222.0.71 () {56 vars in 1135 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:42.431562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.475062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57beac87-57be-4072-8233-6e44e90b4c3d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.519380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.520077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.520356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.520630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.521343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:42.522052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.523019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.528269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 397/792] 10.222.0.71 () {56 vars in 1124 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:42.533784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.553943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603e753-f69b-4f0c-b1f7-264419a8530e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.600134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.601040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.601427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.601811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.613223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.614752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:42.635053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:42.636479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:42.637687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:42.637687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:42.637687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:42.637687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:42.637687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 396/793] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:42.643571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.667810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b713992-c163-46f0-b07f-51f0de03d8f4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.706258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.707021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.707387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.707749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.708525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:42.708894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.710325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.716656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 398/794] 10.222.0.71 () {56 vars in 1124 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:42.722938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.778267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02121a4-bee2-4914-b51e-118336f56274 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.813589 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9dda5-2859-459c-a2c8-fd7b25a229f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.814257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9dda5-2859-459c-a2c8-fd7b25a229f8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.814561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9dda5-2859-459c-a2c8-fd7b25a229f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.815112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9dda5-2859-459c-a2c8-fd7b25a229f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.815596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9dda5-2859-459c-a2c8-fd7b25a229f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:42.815943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9dda5-2859-459c-a2c8-fd7b25a229f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.817603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9dda5-2859-459c-a2c8-fd7b25a229f8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.829389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 397/795] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:42.830858 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9b64-2872-468f-8212-95e79dbacd50 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.831149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9b64-2872-468f-8212-95e79dbacd50 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.831427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9b64-2872-468f-8212-95e79dbacd50 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.833008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 398/796] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:42.837705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9b64-2872-468f-8212-95e79dbacd50 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:42.842409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93925388-b1f7-475c-93ed-a80f77787aad None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.879042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e67fe-9db2-4772-9c06-43aacbda2982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.880029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e67fe-9db2-4772-9c06-43aacbda2982 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.880489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e67fe-9db2-4772-9c06-43aacbda2982 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.880957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e67fe-9db2-4772-9c06-43aacbda2982 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.881874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e67fe-9db2-4772-9c06-43aacbda2982 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.883401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e67fe-9db2-4772-9c06-43aacbda2982 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:42.941887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9b64-2872-468f-8212-95e79dbacd50 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:42.975316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:42.976149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.976485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.976762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.986164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 399/797] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:56:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:42.987855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84774a0-2c33-4c37-b55b-edbf6e7b543a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:42.988683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84774a0-2c33-4c37-b55b-edbf6e7b543a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:42.989248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:42.989485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84774a0-2c33-4c37-b55b-edbf6e7b543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:42.990812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:43.006860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:43.007854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:43.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 399/798] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:43.016429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d84774a0-2c33-4c37-b55b-edbf6e7b543a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:43.018044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d84774a0-2c33-4c37-b55b-edbf6e7b543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:43.020373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:43.116971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 400/799] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:43.153216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84774a0-2c33-4c37-b55b-edbf6e7b543a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:43.188165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df892d52-9d8b-445d-a1ce-da2f1cd7b32d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:43.260407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:43.261093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.261373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.261668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.262452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:43.262759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:43.264252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:43.273522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84774a0-2c33-4c37-b55b-edbf6e7b543a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:43.285986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 400/800] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:43.291295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:43.325632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:43.326498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.326784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.327056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.346118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:43.356197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:43.357163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:43.358441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 401/801] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:43.363683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:43.432322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed494-c4c5-46ea-8a64-ccfbd7751708 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:43.470729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:43.471896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.472326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.472746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/918d67038f9c4d0fa4d2456409245774/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.474919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=918d67038f9c4d0fa4d2456409245774, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:43.476330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ca073-944d-4caa-b80e-2f68057519dc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:43.476717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:43.518231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:43.519345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.519911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.520339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.521237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:43.521734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:43.523275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:43.532954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 402/802] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:43.544852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 401/803] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:43 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/918d67038f9c4d0fa4d2456409245774/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 14:56:43.545770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:43.658722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f69af0-a1c4-4e5a-afd8-96f946028b13 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:43.698761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48df39-63cc-4cce-bc3d-e79ec3293b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:43.699895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48df39-63cc-4cce-bc3d-e79ec3293b2e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.700327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48df39-63cc-4cce-bc3d-e79ec3293b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.700753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48df39-63cc-4cce-bc3d-e79ec3293b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.713830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48df39-63cc-4cce-bc3d-e79ec3293b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:43.715599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48df39-63cc-4cce-bc3d-e79ec3293b2e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3b48df39-63cc-4cce-bc3d-e79ec3293b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:43.734027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:43.735244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:43.736408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:43.736408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:43.736408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:43.736408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:43.738333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 403/804] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:43.744034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:43.853377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c07a8-d1ac-40a7-ac33-7d0345bc40ee None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:43.888608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:43.889565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.890047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.890472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.891356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:43.891853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:43.893242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:43.901211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656598-f702-4109-b890-a7f6e814be58 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.901498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656598-f702-4109-b890-a7f6e814be58 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.901775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656598-f702-4109-b890-a7f6e814be58 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.903452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 404/805] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:43.909249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 402/806] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:43.910871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c239d-e72f-43e4-b74a-d6ddc3a00509 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:43.911222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c239d-e72f-43e4-b74a-d6ddc3a00509 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:43.911508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c239d-e72f-43e4-b74a-d6ddc3a00509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:43.918574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:43.930459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-766c239d-e72f-43e4-b74a-d6ddc3a00509 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:43.932028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-766c239d-e72f-43e4-b74a-d6ddc3a00509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:44.004202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 405/807] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:44.039925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c239d-e72f-43e4-b74a-d6ddc3a00509 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.045672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472ae3-6342-4102-8254-68c4eb0d4d9f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.084341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.085472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.085758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.086034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.095327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.097198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:44.114360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:44.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:44.116403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:44.120247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 403/808] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:56:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:44.121630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.231239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289b7b-a296-4f84-b04e-b7e39ee03cae None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.270107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.270715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.271001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.271277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.272149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:44.272460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.274497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:44.294904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 404/809] 10.222.0.71 () {56 vars in 1149 bytes} [Mon May 16 14:56:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:44.301458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.337746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c239d-e72f-43e4-b74a-d6ddc3a00509 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.409600 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84920b-be9f-4889-bb07-34efa032e208 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.445226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.445882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.446620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.446730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.446936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.447231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.447655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.448129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] PATH_INFO: `/v3/services/network` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.449001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.450585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:44.457694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.459194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:44.483653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:44.485750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 14:56:44.486619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:44.487372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:44.488216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:44.489137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:44.489137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:44.489137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:44.489137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:44.489137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 405/810] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:44.489424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 406/811] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:44 2022] GET /identity/v3/services/network => generated 88 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:44.496290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.496509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.654791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816f615-06ec-4964-a25a-09f57b520a46 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.691053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.691779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.692060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.692332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.693039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] RBAC: Adding query filter params (name=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:44.693340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.694712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:44.701998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2d313-adc4-40c7-bcdb-7978b0e085fb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.707902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 406/812] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:56:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:44.713356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.741795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.742928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.743420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.744033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.744985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:44.745492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.747248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:44.761617 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 407/813] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:44.769667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.887831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a17d8-1ee1-4a7b-b524-d3e2ab6bc1ca None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.932264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6da45-7fa3-47ea-a900-ba28eb88a188 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:44.933673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.934343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.934633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.934908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.935621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] RBAC: Adding query filter params (type=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:44.935949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.938014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:44.951706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 407/814] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:56:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:44.957092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:44.976372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:44.977558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:44.978052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:44.978559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:44.996793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:44.998623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:45.031918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:45.032892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:45.033790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:45.036347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 408/815] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:45.041664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:45.059282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba01818-dd81-4ff4-8a33-88bf6d148dea None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:45.100077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:45.100758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.101040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.101314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.102092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:45.103507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:45.199690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e279ea8-72ff-4800-a816-2f2ba8091dc3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:45.217094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 408/816] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:56:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 14:56:45.261048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f6916-0aea-43f0-b9cc-36974f9a6a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:45.261794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f6916-0aea-43f0-b9cc-36974f9a6a80 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.262083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f6916-0aea-43f0-b9cc-36974f9a6a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.262363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f6916-0aea-43f0-b9cc-36974f9a6a80 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.263167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f6916-0aea-43f0-b9cc-36974f9a6a80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:45.263476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f6916-0aea-43f0-b9cc-36974f9a6a80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:45.265028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f6916-0aea-43f0-b9cc-36974f9a6a80 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:45.284655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 409/817] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:45.290587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:45.333042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c283c-9778-4267-ab68-f8d60f0603d6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.333390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c283c-9778-4267-ab68-f8d60f0603d6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.336173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c283c-9778-4267-ab68-f8d60f0603d6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.337606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 410/818] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:45.349569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69504be1-e103-4007-844a-5eeb9c898592 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.350181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69504be1-e103-4007-844a-5eeb9c898592 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.350181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69504be1-e103-4007-844a-5eeb9c898592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.383048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-69504be1-e103-4007-844a-5eeb9c898592 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:45.384299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-69504be1-e103-4007-844a-5eeb9c898592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:45.458468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1622da5-b0e4-443c-8cc3-21eac2a84cdd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:45.511413 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 411/819] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:45.514246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:45.515965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.516464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.516963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/4dd72a14691c48dd95845b7b46eb8c58/roles/29b60e8912a74b2dafcb91fbb14ee519` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.538498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69504be1-e103-4007-844a-5eeb9c898592 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:45.544699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=4dd72a14691c48dd95845b7b46eb8c58, role_id=29b60e8912a74b2dafcb91fbb14ee519)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:45.546152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:45.623776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 409/820] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:45 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/4dd72a14691c48dd95845b7b46eb8c58/roles/29b60e8912a74b2dafcb91fbb14ee519 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:56:45.653402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69504be1-e103-4007-844a-5eeb9c898592 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:45.693684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:45.694509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.694795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.695051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.708124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:45.716394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:45.717371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:45.719071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 412/821] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:45.725239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:45.842481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821020bf-bf4b-4209-955a-8a385abd45ea None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:45.882966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7514d-a6fe-4009-bed3-1c8f0286278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:45.883653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7514d-a6fe-4009-bed3-1c8f0286278b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.883936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7514d-a6fe-4009-bed3-1c8f0286278b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.884206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7514d-a6fe-4009-bed3-1c8f0286278b None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.884954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7514d-a6fe-4009-bed3-1c8f0286278b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:45.885251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7514d-a6fe-4009-bed3-1c8f0286278b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:45.886627 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7514d-a6fe-4009-bed3-1c8f0286278b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:45.895451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 410/822] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:45.900599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:45.958422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5a651-6151-4e3c-802d-ac0d6452708c None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.958706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5a651-6151-4e3c-802d-ac0d6452708c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.958987 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5a651-6151-4e3c-802d-ac0d6452708c None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.960619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 411/823] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:45.965523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cb15e-e765-44db-8beb-d698e3560de3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:45.965807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cb15e-e765-44db-8beb-d698e3560de3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:45.966084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cb15e-e765-44db-8beb-d698e3560de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:45.985814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-960cb15e-e765-44db-8beb-d698e3560de3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:45.986992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-960cb15e-e765-44db-8beb-d698e3560de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:46.002230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76b769-787f-475c-9a58-4b8a33122e76 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.045652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.046564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.047020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.047370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.061974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.064382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.071717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 412/824] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:46.087824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:46.089653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:46.090788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:46.090788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:46.090788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:46.090788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:46.091067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cb15e-e765-44db-8beb-d698e3560de3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.095653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 413/825] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:46.098741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.212657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cb15e-e765-44db-8beb-d698e3560de3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.221158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf92bd-b387-419c-9cda-5b006f73c037 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.249474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbddda-0b1a-4697-8bef-4a9ab1acc16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.250723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbddda-0b1a-4697-8bef-4a9ab1acc16a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.251022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbddda-0b1a-4697-8bef-4a9ab1acc16a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.251323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbddda-0b1a-4697-8bef-4a9ab1acc16a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.252003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbddda-0b1a-4697-8bef-4a9ab1acc16a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.253381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbddda-0b1a-4697-8bef-4a9ab1acc16a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.255044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0c1eb-1211-4481-a368-d86e55802bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.255738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0c1eb-1211-4481-a368-d86e55802bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.256023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0c1eb-1211-4481-a368-d86e55802bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.256299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0c1eb-1211-4481-a368-d86e55802bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.257018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0c1eb-1211-4481-a368-d86e55802bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:46.257324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0c1eb-1211-4481-a368-d86e55802bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.258750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0c1eb-1211-4481-a368-d86e55802bb0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-57bbddda-0b1a-4697-8bef-4a9ab1acc16a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:46.266953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:46.268023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:46.268873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:46.269636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 413/826] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:46.270363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9123013-851a-4a0d-8857-bc08bbcbb976 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.270643 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9123013-851a-4a0d-8857-bc08bbcbb976 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.270916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9123013-851a-4a0d-8857-bc08bbcbb976 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.272354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 414/827] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:46.276649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 414/828] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:46.276881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9123013-851a-4a0d-8857-bc08bbcbb976 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.277898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5c3d-b5fd-4a72-bcb7-b61949d619af None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.278184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5c3d-b5fd-4a72-bcb7-b61949d619af None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.278505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5c3d-b5fd-4a72-bcb7-b61949d619af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.296798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e2d5c3d-b5fd-4a72-bcb7-b61949d619af None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:46.298136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e2d5c3d-b5fd-4a72-bcb7-b61949d619af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:46.363528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 415/829] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:46.364496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5c3d-b5fd-4a72-bcb7-b61949d619af None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.397649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9123013-851a-4a0d-8857-bc08bbcbb976 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.435500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.436520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.436964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.438135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.439020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:46.439485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.441029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.450089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 415/830] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:46.450089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.466275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5c3d-b5fd-4a72-bcb7-b61949d619af None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.500131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.500843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.501128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.501405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.509505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.511053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:46.525667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:46.526644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 416/831] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:46.528405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.549678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d9998-499d-4205-b09a-3177fb23287e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.584150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.584926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.585198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.585456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.596154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:46.603753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:46.604679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:46.605557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 416/832] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:46.606253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.633142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715e207-7b34-4979-86c9-3a58f2986988 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.666479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.667139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.667419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.667713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.668412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:46.668715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.670072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.679664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 417/833] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:46.679664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.704370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240b94c-9b5e-4020-8c3f-e0e7e64812ec None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.740756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.741947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.742850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.743336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.744853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:46.745561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.747658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.783838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 417/834] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:46.788981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.814851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6e3c-1b52-4606-b0e0-ef9352995067 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.862620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.863414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.864383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.875145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:46.875145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:46.875145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:46.876397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:46.887511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 418/835] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:46.893473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:46.935670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c9d8a-bb78-4cd1-980d-10ed5685f9fc None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:46.985867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:46.986896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:46.987189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:46.987464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.008823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79163e-8a08-4f0f-97bb-857016f6b7b4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.042801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.043520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.043821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.044098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.055066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.056683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:47.074943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:47.076130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:47.077165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:47.077165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:47.077165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:47.077165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:47.080052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.081099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 419/836] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:47.081927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.082047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.190630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 418/837] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 14:56:47.191496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.202009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f275b8-6534-4ce6-a954-d8ea4583562f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.246349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.247272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.248178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.248696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.249880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:47.250340 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.252647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.268971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 420/838] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:47.274103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.301890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd74752-36dc-425d-b388-0bd0c573b5b7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.348296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.349341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.349802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.350225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.363385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.365491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.376930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636e2dc-3c27-43c3-acdd-0824b95de837 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:47.390423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:47.392250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:47.393389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:47.393389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:47.393389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:47.393389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:47.396011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 419/839] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:47.401092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.421164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.423101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.423359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.423752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.431500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.433096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:47.447454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:47.448647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 14:56:47.449903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:47.456007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 421/840] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:47.457351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.522963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a3a7-cf99-4567-a678-f712809cddae None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.556713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0009c4-1853-4881-a47e-334e63965bc5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.557611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.558441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.558888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.559323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.560304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:47.560822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.562966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.576120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 420/841] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:47.583281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.601137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.601833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.602153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.602433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.603875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:47.604263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.606027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.632591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 422/842] 10.222.0.71 () {56 vars in 1135 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:47.633636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.696785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044f58-07a8-4d70-8be0-5cee6c33c8e2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.736050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.736927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.737238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.737556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.745205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.746752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:47.762743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:47.763752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:47.764638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:47.766194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43313e-6c86-4500-bb81-7baf3ccab20e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.769657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 421/843] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:47.769783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.804696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18274278-7f6d-4b71-b0cb-b661eec1fec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.805749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18274278-7f6d-4b71-b0cb-b661eec1fec0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.806232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18274278-7f6d-4b71-b0cb-b661eec1fec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.806701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18274278-7f6d-4b71-b0cb-b661eec1fec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.873406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4c4fd-c6c5-4618-8f7f-b53877480cf7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:47.888166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18274278-7f6d-4b71-b0cb-b661eec1fec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.890061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18274278-7f6d-4b71-b0cb-b661eec1fec0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.908664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa5213-3bb0-4b5f-827b-cb15f220e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:47.909328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa5213-3bb0-4b5f-827b-cb15f220e32e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.909608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa5213-3bb0-4b5f-827b-cb15f220e32e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.909967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa5213-3bb0-4b5f-827b-cb15f220e32e None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.910671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa5213-3bb0-4b5f-827b-cb15f220e32e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:47.910980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa5213-3bb0-4b5f-827b-cb15f220e32e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:47.912442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa5213-3bb0-4b5f-827b-cb15f220e32e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:47.925209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 422/844] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:47.926800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efc795-409a-4a38-8a15-93b84f278abb None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.927289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efc795-409a-4a38-8a15-93b84f278abb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.927565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efc795-409a-4a38-8a15-93b84f278abb None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:47.928939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 423/845] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:47.932483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efc795-409a-4a38-8a15-93b84f278abb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:47.996105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 423/846] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:56:47.997729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15895162-c7eb-438d-a994-61953d522a41 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:47.998216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15895162-c7eb-438d-a994-61953d522a41 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:47.998687 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15895162-c7eb-438d-a994-61953d522a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.015112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-15895162-c7eb-438d-a994-61953d522a41 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:48.016657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-15895162-c7eb-438d-a994-61953d522a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:48.044447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efc795-409a-4a38-8a15-93b84f278abb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.079640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.080315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.080690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.081065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.081882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=3d58a6e2eac047ad8e88a1f973ecc697) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:48.082189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.082743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 424/847] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:48.083617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.088553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15895162-c7eb-438d-a994-61953d522a41 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.091676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 424/848] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:47 2022] GET /identity/v3/role_assignments?user.id=3d58a6e2eac047ad8e88a1f973ecc697&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:48.106460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.202623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15895162-c7eb-438d-a994-61953d522a41 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.237133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218ff7-e1dc-497e-bc50-543daaa3d6ac None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.238579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.239393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.239692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.239964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.250665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.252205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:48.271472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:48.272479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:48.273358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:48.273358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:48.273358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:48.273358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:48.277430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 425/849] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:48.278403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.290157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.291777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.292572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.293363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.309983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:48.318893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:48.327504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 425/850] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:48.328944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.411163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6217-0c91-4a6d-99e4-9be0da851c2f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.445230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449157-e737-4c45-b430-b1875fa8af4b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.451296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.452045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.452327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.452597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.453869 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:48.454173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.456023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.471676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 426/851] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:48.477912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.487964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.488779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.489160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.489532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.490386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:48.491208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.492309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.501926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 426/852] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:48.506838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.604970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e22485-e795-450a-af94-7f9c554adae0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.606823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abbe8-0253-4a54-b4fe-bb71e25551f6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.641014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.642131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.642457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.642832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.647771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.648744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.649177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.649589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.653732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.655274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.659272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.660916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:48.673849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:48.674921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:48.676049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:48.676049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:48.676049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:48.676049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:48.677551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 427/853] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:48.679525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:48.680567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:48.681637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:48.682904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.688520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 427/854] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:48.690036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.800815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bbb5a-7a9e-4647-8c93-bf198d5c7e11 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.808556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7311587-3f0e-49d9-8d3f-5dd6af50929d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.846966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.847643 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:48.847879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.848257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.848626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.848921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:48.849335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:48.849707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:48.849876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:48.850184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.850885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:48.851335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:48.851910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.852998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:48.867668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 428/855] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:48.871041 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.873460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 428/856] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:48.882558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:48.971128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9f8f0-0f2d-4367-9f97-edc574f739bb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:48.987294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fb3ad-67c1-477d-a47e-c650c6e20306 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:49.006031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f102d6-1570-4687-94eb-9333099279a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:49.006974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f102d6-1570-4687-94eb-9333099279a9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.007432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f102d6-1570-4687-94eb-9333099279a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.007916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f102d6-1570-4687-94eb-9333099279a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.008945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f102d6-1570-4687-94eb-9333099279a9 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=918d67038f9c4d0fa4d2456409245774) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:49.009441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f102d6-1570-4687-94eb-9333099279a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:49.011032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f102d6-1570-4687-94eb-9333099279a9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:49.017266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 429/857] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/role_assignments?user.id=918d67038f9c4d0fa4d2456409245774&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:49.018699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ab212-cd4e-47ac-bb1e-4cd7ff5420ca None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.018984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ab212-cd4e-47ac-bb1e-4cd7ff5420ca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.019255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ab212-cd4e-47ac-bb1e-4cd7ff5420ca None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.020726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 430/858] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:49.023999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:49.025151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.025687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.026167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.029342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ea37-9c32-4f99-91a8-48d7aca33cfa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.029622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ea37-9c32-4f99-91a8-48d7aca33cfa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.029897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ea37-9c32-4f99-91a8-48d7aca33cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.036958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:49.038841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:49.048149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9be6ea37-9c32-4f99-91a8-48d7aca33cfa None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:49.049358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9be6ea37-9c32-4f99-91a8-48d7aca33cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:49.056835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:49.058235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:49.059199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:49.062017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 429/859] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:56:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:49.068072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:49.136038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 431/860] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:49.152572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6ea37-9c32-4f99-91a8-48d7aca33cfa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:49.179563 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b706-5073-44b5-86af-2bd9f49ff898 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:49.217930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:49.218989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.219274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.219551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.221417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:49.221768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:49.223269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:49.258016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 430/861] 10.222.0.71 () {56 vars in 1149 bytes} [Mon May 16 14:56:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:49.269270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:49.275660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6ea37-9c32-4f99-91a8-48d7aca33cfa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:49.325982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:49.335615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.335615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.335615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.335615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:49.335615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:49.398383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27790638-b3a0-4ccd-a1a1-6f7170981473 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:49.440584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 432/862] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:56:49 2022] POST /identity/v3/services => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 14:56:49.441165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:49.441870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.442192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.442466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.518413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:49.519838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:49.623606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 431/863] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:49 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:56:49.633792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:49.796811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac9fb2-cb80-4281-9444-847112a941bb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:49.832741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:49.834072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:49.834582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:49.835084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:49.848567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:49.850331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:49.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:49.868962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:49.869939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:49.869939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:49.869939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:49.869939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:49.872272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 433/864] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:49.883435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:49.978187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906469b-ffaa-4f8c-b4d8-3420839ab22a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:50.012663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:50.013333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:50.013610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:50.013881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:50.014600 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:50.014902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:50.016815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:50.028594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 432/865] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:50.034525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:50.138428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25064465-3796-407e-af64-bb62479da4b0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:50.176095 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:50.176931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:50.177209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:50.177481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:50.184286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:50.185722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:50.200273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:50.201199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:50.203239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 434/866] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:50.207319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:50.307327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b1188-e925-45fb-918c-a0ba8892c6aa None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:50.343370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f844cd1-80a2-40eb-bd6a-3d630f89ad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:50.344613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f844cd1-80a2-40eb-bd6a-3d630f89ad78 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:50.344926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f844cd1-80a2-40eb-bd6a-3d630f89ad78 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:50.345140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f844cd1-80a2-40eb-bd6a-3d630f89ad78 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:50.345929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f844cd1-80a2-40eb-bd6a-3d630f89ad78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:50.346234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f844cd1-80a2-40eb-bd6a-3d630f89ad78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:50.347694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f844cd1-80a2-40eb-bd6a-3d630f89ad78 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:50.363533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 433/867] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:50.369749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:50.475978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a9d99-269c-434e-9c78-d65f1eaaa445 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:50.511986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e98041-b1bd-4513-aea0-8ceec69da1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:50.513014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e98041-b1bd-4513-aea0-8ceec69da1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:50.513510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e98041-b1bd-4513-aea0-8ceec69da1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:50.513991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e98041-b1bd-4513-aea0-8ceec69da1a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:50.515033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e98041-b1bd-4513-aea0-8ceec69da1a9 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=4dd72a14691c48dd95845b7b46eb8c58) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:50.516293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e98041-b1bd-4513-aea0-8ceec69da1a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:50.517961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e98041-b1bd-4513-aea0-8ceec69da1a9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:50.524639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 435/868] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:50 2022] GET /identity/v3/role_assignments?user.id=4dd72a14691c48dd95845b7b46eb8c58&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:50.563004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93d8c3-0302-4c2f-b635-fd8f129369e4 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:50.563339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93d8c3-0302-4c2f-b635-fd8f129369e4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:50.563522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93d8c3-0302-4c2f-b635-fd8f129369e4 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:50.565171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 434/869] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:50.570873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f20b-607a-42a0-b606-deea58f984b7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:50.571381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f20b-607a-42a0-b606-deea58f984b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:50.571859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f20b-607a-42a0-b606-deea58f984b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:50.589633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4765f20b-607a-42a0-b606-deea58f984b7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:50.591126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4765f20b-607a-42a0-b606-deea58f984b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:50.667766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 436/870] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:50.688594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93d8c3-0302-4c2f-b635-fd8f129369e4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:50.803172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93d8c3-0302-4c2f-b635-fd8f129369e4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:50.837570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:50.838381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:50.838656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:50.838921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:50.839580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:50.842767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:50.855968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:50.856993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:50.857986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 435/871] 10.222.0.71 () {56 vars in 1132 bytes} [Mon May 16 14:56:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:50.866069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765f20b-607a-42a0-b606-deea58f984b7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:50.971763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765f20b-607a-42a0-b606-deea58f984b7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:51.007924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077662c-0b1e-47f2-ba3f-bd798030950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:51.009013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077662c-0b1e-47f2-ba3f-bd798030950c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.009511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077662c-0b1e-47f2-ba3f-bd798030950c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.009992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077662c-0b1e-47f2-ba3f-bd798030950c None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.010902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077662c-0b1e-47f2-ba3f-bd798030950c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:51.011406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077662c-0b1e-47f2-ba3f-bd798030950c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:51.013004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077662c-0b1e-47f2-ba3f-bd798030950c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:51.023304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 437/872] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:56:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:51.024977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:51.124345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658fc31-f1fc-441e-8a30-27952d54785b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:51.159153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74754d-6218-496a-8503-e329ca8fff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:51.159843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74754d-6218-496a-8503-e329ca8fff89 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.160122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74754d-6218-496a-8503-e329ca8fff89 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.160397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74754d-6218-496a-8503-e329ca8fff89 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.161102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74754d-6218-496a-8503-e329ca8fff89 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:51.161408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74754d-6218-496a-8503-e329ca8fff89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:51.162764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74754d-6218-496a-8503-e329ca8fff89 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:51.167585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 436/873] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:56:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:51.475888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b21b0-0992-4d7c-bb69-392616dafadf None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.476294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b21b0-0992-4d7c-bb69-392616dafadf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.476854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b21b0-0992-4d7c-bb69-392616dafadf None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.478042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 438/874] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:51.483475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3646d-c2be-4d23-83ba-e7fdd079fd36 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.483921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3646d-c2be-4d23-83ba-e7fdd079fd36 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.484102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3646d-c2be-4d23-83ba-e7fdd079fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.502810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c3d3646d-c2be-4d23-83ba-e7fdd079fd36 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:51.504034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c3d3646d-c2be-4d23-83ba-e7fdd079fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:51.570936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 437/875] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:51.592555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b21b0-0992-4d7c-bb69-392616dafadf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:51.695112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b21b0-0992-4d7c-bb69-392616dafadf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:51.733649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086253ca-b570-4e8e-a2c6-e6520ea218fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:51.735290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086253ca-b570-4e8e-a2c6-e6520ea218fc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.735827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086253ca-b570-4e8e-a2c6-e6520ea218fc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.736197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086253ca-b570-4e8e-a2c6-e6520ea218fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.736861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086253ca-b570-4e8e-a2c6-e6520ea218fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:51.738299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086253ca-b570-4e8e-a2c6-e6520ea218fc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-086253ca-b570-4e8e-a2c6-e6520ea218fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:51.751804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:51.752831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:51.753940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 439/876] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:56:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:51.759285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3646d-c2be-4d23-83ba-e7fdd079fd36 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:51.864558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3646d-c2be-4d23-83ba-e7fdd079fd36 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:51.889274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282be65-e095-43c0-b656-4a3a828e1a53 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.890048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282be65-e095-43c0-b656-4a3a828e1a53 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.890650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282be65-e095-43c0-b656-4a3a828e1a53 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.892664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 440/877] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:51.898803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b913a0-ea80-405b-b657-1feb7bdea168 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.898803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b913a0-ea80-405b-b657-1feb7bdea168 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.898803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b913a0-ea80-405b-b657-1feb7bdea168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.899088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:51.899848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:51.899848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:51.900160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:51.900867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:51.901263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:51.902837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:51.909680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 438/878] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:56:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:51.915903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:51.916068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-24b913a0-ea80-405b-b657-1feb7bdea168 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:51.917075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-24b913a0-ea80-405b-b657-1feb7bdea168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:51.991960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 441/879] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:52.008099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b913a0-ea80-405b-b657-1feb7bdea168 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:52.016281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e661abd-003f-42fa-bafa-d53dfc60f9da None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:52.051021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:52.051720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:52.052077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:52.052352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:52.053063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:52.053363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:52.054733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:52.059711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 439/880] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:56:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:52.118267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b913a0-ea80-405b-b657-1feb7bdea168 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:52.156369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:52.157469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:52.157749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:52.158023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:52.158729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:52.160159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:52.173472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:52.176030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:52.178165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 442/881] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:52.184175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:52.284523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9ffa1-cb27-4eb3-bfc6-514d094e2d1b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:52.317593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:52.329204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:52.329204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:52.329204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:52.329204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:52.329204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:52.329204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:52.330128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 440/882] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:52.332259 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:52.435868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989734-1366-4a03-83d3-9cf56aee4843 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:52.471673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:52.472348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:52.472627 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:52.472899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:52.473603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:52.473906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:52.475269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:52.480703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 443/883] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:52.486430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:52.586836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8f11-a241-494a-82cc-8b03630b5640 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:52.619661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6db1f-ff6c-4761-98ca-e34607a5f2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:52.620321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6db1f-ff6c-4761-98ca-e34607a5f2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:52.620608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6db1f-ff6c-4761-98ca-e34607a5f2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:52.620885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6db1f-ff6c-4761-98ca-e34607a5f2a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:52.621680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6db1f-ff6c-4761-98ca-e34607a5f2a8 None admin] RBAC: Adding query filter params (interface=public, service_id=1827a7aec7db4a5b88173ebf00a799c3, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:52.621989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6db1f-ff6c-4761-98ca-e34607a5f2a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:52.623352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6db1f-ff6c-4761-98ca-e34607a5f2a8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:52.629475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 441/884] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 14:56:52 2022] GET /identity/v3/endpoints?service_id=1827a7aec7db4a5b88173ebf00a799c3&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:52.634484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:52.733849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9ccb6-7a22-4981-9d37-8a142b5a0b78 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:52.768096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5f62d-24f1-4576-9699-7e89564275c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:52.768763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f62d-24f1-4576-9699-7e89564275c2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:52.769042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f62d-24f1-4576-9699-7e89564275c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:52.769316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f62d-24f1-4576-9699-7e89564275c2 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:52.769991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5f62d-24f1-4576-9699-7e89564275c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:52.771352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5f62d-24f1-4576-9699-7e89564275c2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:52.777091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 444/885] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:56:52 2022] GET /identity/v3/services => generated 1011 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 14:56:53.009993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9238328-d49a-4381-852f-a0ba3d7f4553 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.010719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9238328-d49a-4381-852f-a0ba3d7f4553 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.011223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9238328-d49a-4381-852f-a0ba3d7f4553 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.013111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 442/886] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:53.019699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a613caa-31ae-43cc-8226-46acb432ea5f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.020257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a613caa-31ae-43cc-8226-46acb432ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.020790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a613caa-31ae-43cc-8226-46acb432ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.039823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4a613caa-31ae-43cc-8226-46acb432ea5f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:53.041453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a613caa-31ae-43cc-8226-46acb432ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:53.118058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 445/887] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:53.134462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9238328-d49a-4381-852f-a0ba3d7f4553 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:53.244810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9238328-d49a-4381-852f-a0ba3d7f4553 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:53.278995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:53.279867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.280152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.280423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.291221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:53.300149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:53.301114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:53.302470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 443/888] 10.222.0.71 () {56 vars in 1120 bytes} [Mon May 16 14:56:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:53.309397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a613caa-31ae-43cc-8226-46acb432ea5f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:53.416326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a613caa-31ae-43cc-8226-46acb432ea5f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:53.450457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d1cd2-fe09-4b2e-927d-048ea8d40016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:53.451127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1cd2-fe09-4b2e-927d-048ea8d40016 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.451406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1cd2-fe09-4b2e-927d-048ea8d40016 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.451699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1cd2-fe09-4b2e-927d-048ea8d40016 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.452434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d1cd2-fe09-4b2e-927d-048ea8d40016 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:53.452734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d1cd2-fe09-4b2e-927d-048ea8d40016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:53.454114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d1cd2-fe09-4b2e-927d-048ea8d40016 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:53.462417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 446/889] 10.222.0.71 () {56 vars in 1115 bytes} [Mon May 16 14:56:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:53.467993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:53.571701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16dd7f-5616-41c1-a9d6-9296649cd0cb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:53.573453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a2fb-fe2a-452e-b04d-b3c2bb9b51c8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.574223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a2fb-fe2a-452e-b04d-b3c2bb9b51c8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.574711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a2fb-fe2a-452e-b04d-b3c2bb9b51c8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.580470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 447/890] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:53.582296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3ecf8-743e-43e3-9967-e863ed2cc87a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.582834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3ecf8-743e-43e3-9967-e863ed2cc87a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.583343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3ecf8-743e-43e3-9967-e863ed2cc87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.600847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-63f3ecf8-743e-43e3-9967-e863ed2cc87a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:53.602487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-63f3ecf8-743e-43e3-9967-e863ed2cc87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:53.606409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:53.607201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.607476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.607836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.619205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:53.620677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:53.638141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:53.639075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:53.640020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:53.640020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:53.640020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:53.640020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:53.645425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 444/891] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:53.648337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:53.695731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 448/892] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:53.711135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3ecf8-743e-43e3-9967-e863ed2cc87a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:53.753975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f75e2-9a13-4c84-8b8d-82f811cf6f15 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:53.788545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:53.789578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.790245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.790720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.791654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:53.792159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:53.793730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:53.811512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 445/893] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:53.813802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:53.829322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3ecf8-743e-43e3-9967-e863ed2cc87a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:53.874867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:53.876135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.876403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.876675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.877403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:53.878773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:53.929729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90369e-e75f-4006-8da5-65619bce0f24 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:53.966372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:53.967311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:53.967670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:53.975658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:53.975898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 449/894] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:56:53 2022] POST /identity/v3/services => generated 269 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 14:56:53.983054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:53.985375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:54.003296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:54.004391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:54.005370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:54.008912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 446/895] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:56:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:54.011044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:54.131193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eb1b8-3810-446d-a8dd-f59ab89ff690 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:54.169348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a24ff-d287-4b99-a03c-dd365c22c8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:54.170221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a24ff-d287-4b99-a03c-dd365c22c8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:54.170508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a24ff-d287-4b99-a03c-dd365c22c8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:54.170784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a24ff-d287-4b99-a03c-dd365c22c8d2 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:54.171730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a24ff-d287-4b99-a03c-dd365c22c8d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:54.171984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a24ff-d287-4b99-a03c-dd365c22c8d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:54.173625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a24ff-d287-4b99-a03c-dd365c22c8d2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:54.195801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 450/896] 10.222.0.71 () {56 vars in 1149 bytes} [Mon May 16 14:56:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:54.199857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:54.304773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682c40-89d7-4311-ad72-eb243ffb61f9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:54.339213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:54.340274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:54.340783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:54.341313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:54.357727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d2594-90ae-41c2-baa1-4c19f8b1b381 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:54.358008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d2594-90ae-41c2-baa1-4c19f8b1b381 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:54.363762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d2594-90ae-41c2-baa1-4c19f8b1b381 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:54.363762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 451/897] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:54.365211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b637ce-39e1-4968-bcbc-a80940ccb5c4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:54.365500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b637ce-39e1-4968-bcbc-a80940ccb5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:54.365783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b637ce-39e1-4968-bcbc-a80940ccb5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:54.383971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-09b637ce-39e1-4968-bcbc-a80940ccb5c4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:54.385153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09b637ce-39e1-4968-bcbc-a80940ccb5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:54.430172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:54.432282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:54.459761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 452/898] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:54.478209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b637ce-39e1-4968-bcbc-a80940ccb5c4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:54.551782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 447/899] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:56:54 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:56:54.557372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:54.590220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b637ce-39e1-4968-bcbc-a80940ccb5c4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:54.631546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:54.632497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:54.632791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:54.633047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:54.633784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:54.635845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:54.670025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ac84f-32a2-44db-962b-9283996e85d7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:54.702966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:54.704765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:54.705106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:54.705385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:54.718764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:54.721080 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:54.723189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 453/900] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:56:54 2022] POST /identity/v3/services => generated 241 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:54.741577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:54.742559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:54.743460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:54.743460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:54.743460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:54.743460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:54.745977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 448/901] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:54.752296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:54.870932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a9832-54ae-424b-a352-c47345efc800 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:54.909853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:54.910934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:54.911413 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:54.911913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:54.912881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:54.913432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:54.915104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:54.929828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 454/902] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:54.940923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:55.047080 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3ac71-e89d-4076-80f4-8a78dab1e846 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:55.081539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a569c5-428c-4c27-be04-ca62887517ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:55.082338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a569c5-428c-4c27-be04-ca62887517ff None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.082618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a569c5-428c-4c27-be04-ca62887517ff None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.082889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a569c5-428c-4c27-be04-ca62887517ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.089607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a569c5-428c-4c27-be04-ca62887517ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:55.091045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a569c5-428c-4c27-be04-ca62887517ff None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c0a569c5-428c-4c27-be04-ca62887517ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:55.105733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:55.106626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:55.107538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:55.111801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 449/903] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:55.116853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:55.123369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b00be-455f-484e-9346-295f05bfedc8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.123604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b00be-455f-484e-9346-295f05bfedc8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.123836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b00be-455f-484e-9346-295f05bfedc8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.126016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 450/904] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:55.131793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542ce7-a633-49f7-9832-601282d05dfd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.132082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542ce7-a633-49f7-9832-601282d05dfd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.132364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542ce7-a633-49f7-9832-601282d05dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.153036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-46542ce7-a633-49f7-9832-601282d05dfd None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:55.154212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-46542ce7-a633-49f7-9832-601282d05dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:55.232521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 451/905] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:55.247550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecea98-3abc-4408-8624-bd2bcdddac7d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:55.259084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46542ce7-a633-49f7-9832-601282d05dfd None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:55.288937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:55.290036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.290441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.290722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.291513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:55.291863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:55.294327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:55.311735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 455/906] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:55.316242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:55.374155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46542ce7-a633-49f7-9832-601282d05dfd None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:55.410844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:55.411932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.412216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.412492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.413303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:55.415495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:55.431164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:55.432424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:55.433295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 452/907] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:55.438789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:55.451599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1457a3-6695-43df-b0b9-a0a645aa0f25 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:55.487833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:55.488583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.488957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.489304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.490569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] RBAC: Adding query filter params (role.id=84e2b91dd928402ab1eb6f28811b0ab3, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=4dd72a14691c48dd95845b7b46eb8c58) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:55.491181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:55.492997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:55.503946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 456/908] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:56:55 2022] GET /identity/v3/role_assignments?user.id=4dd72a14691c48dd95845b7b46eb8c58&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=84e2b91dd928402ab1eb6f28811b0ab3 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:56:55.556253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09069efd-1080-44a0-b274-f8f10798d564 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:55.593814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:55.594591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.594869 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.595141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.595893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:55.596195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:55.597586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:55.603275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 453/909] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:55.608769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:55.720102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e73d8-fe9d-46e5-b0b1-0d1e0f2ec17e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:55.753721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53707f5-23d6-4145-8288-1e30c9fc4a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:55.754390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53707f5-23d6-4145-8288-1e30c9fc4a45 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.754667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53707f5-23d6-4145-8288-1e30c9fc4a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.754936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53707f5-23d6-4145-8288-1e30c9fc4a45 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.755653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53707f5-23d6-4145-8288-1e30c9fc4a45 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:55.755964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53707f5-23d6-4145-8288-1e30c9fc4a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:55.757321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53707f5-23d6-4145-8288-1e30c9fc4a45 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:55.762480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 457/910] 10.222.0.71 () {56 vars in 1114 bytes} [Mon May 16 14:56:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:55.768752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:55.870940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c70bfc-db13-409c-8dab-3050da9e0c04 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:55.904322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a684f-9b3c-408e-9b0e-b1956c63e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:55.904986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a684f-9b3c-408e-9b0e-b1956c63e0a9 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:55.905268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a684f-9b3c-408e-9b0e-b1956c63e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:55.905542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a684f-9b3c-408e-9b0e-b1956c63e0a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:55.906284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a684f-9b3c-408e-9b0e-b1956c63e0a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:55.908250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a684f-9b3c-408e-9b0e-b1956c63e0a9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:55.990490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 454/911] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:56:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:56:56.370107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d9d6-59b6-494b-85a5-113fa708e8eb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:56.370107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d9d6-59b6-494b-85a5-113fa708e8eb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:56.370107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d9d6-59b6-494b-85a5-113fa708e8eb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:56.371347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 458/912] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:56.379510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a90112-199a-4b84-8bdd-3b282168e858 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:56.379849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a90112-199a-4b84-8bdd-3b282168e858 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:56.380132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a90112-199a-4b84-8bdd-3b282168e858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:56.398934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-84a90112-199a-4b84-8bdd-3b282168e858 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:56.400417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-84a90112-199a-4b84-8bdd-3b282168e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:56.473133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 455/913] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:56.495187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d9d6-59b6-494b-85a5-113fa708e8eb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:56.600352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d9d6-59b6-494b-85a5-113fa708e8eb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:56.633213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:56.634165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:56.634630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:56.635149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:56.636079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:56.637444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:56.650319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:56.651288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:56.652160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 459/914] 10.222.0.71 () {56 vars in 1132 bytes} [Mon May 16 14:56:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:56.662644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a90112-199a-4b84-8bdd-3b282168e858 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:56.762710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a90112-199a-4b84-8bdd-3b282168e858 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:56.798022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:56.798772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:56.798957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:56.799225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:56.799965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:56.800267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:56.801629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:56.806906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 456/915] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:56:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:56.812710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:56.928814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef36e7-2c0c-4d61-a41a-a483d09dda42 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:56.965217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509c6eb-7c16-4cd3-b490-684f5e992add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:56.965888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c6eb-7c16-4cd3-b490-684f5e992add None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:56.966172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c6eb-7c16-4cd3-b490-684f5e992add None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:56.966447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c6eb-7c16-4cd3-b490-684f5e992add None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:56.967155 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509c6eb-7c16-4cd3-b490-684f5e992add None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:56.967466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509c6eb-7c16-4cd3-b490-684f5e992add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:56.969159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509c6eb-7c16-4cd3-b490-684f5e992add None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:56.974466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 460/916] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:56:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:56.980069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:57.080895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b0685-b6fc-4001-9372-54b6a8a89f83 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:57.108788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ac42-12f9-4f49-a994-414a90fa4d1d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.109159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ac42-12f9-4f49-a994-414a90fa4d1d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.109508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ac42-12f9-4f49-a994-414a90fa4d1d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.111695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 461/917] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:57.116569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695e3e2-9681-4e0e-8917-de8bb2e7513e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.116929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695e3e2-9681-4e0e-8917-de8bb2e7513e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.117405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695e3e2-9681-4e0e-8917-de8bb2e7513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.124952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:57.125885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.126355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.126807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.127815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] RBAC: Adding query filter params (interface=public, service_id=30f7960becf14fabb26e0ae4125493d9, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:57.128330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:57.130595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:57.134234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9695e3e2-9681-4e0e-8917-de8bb2e7513e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:57.135614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9695e3e2-9681-4e0e-8917-de8bb2e7513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:57.136699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 457/918] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 14:56:56 2022] GET /identity/v3/endpoints?service_id=30f7960becf14fabb26e0ae4125493d9&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:57.143872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:57.235777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 462/919] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:57.248772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4c95f-075d-46f4-a1f6-f1a31d98f77c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:57.255866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695e3e2-9681-4e0e-8917-de8bb2e7513e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:57.292465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:57.293141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.293429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.293708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.294399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:57.295796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:57.304793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 458/920] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:56:57 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 14:56:57.364792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695e3e2-9681-4e0e-8917-de8bb2e7513e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:57.403624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:57.404807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.405306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.405830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.407239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:57.408964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:57.422336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:57.423439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:56:57.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:57.426940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 463/921] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:56:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:57.433386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:57.541486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9419-8c7a-4011-a1f9-29b78faacf21 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:57.574092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:57.574754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.575034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.575307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.576036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:57.576336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:57.577753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:57.583333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 459/922] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:56:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:57.588763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:57.692630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfef51-7237-4e8e-a697-21145fe16d03 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:57.732198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fb709-6d58-40de-8ef8-03f3d1744730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:57.733244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fb709-6d58-40de-8ef8-03f3d1744730 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.733746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fb709-6d58-40de-8ef8-03f3d1744730 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.734231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fb709-6d58-40de-8ef8-03f3d1744730 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.735864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fb709-6d58-40de-8ef8-03f3d1744730 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:57.736401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fb709-6d58-40de-8ef8-03f3d1744730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:57.737970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fb709-6d58-40de-8ef8-03f3d1744730 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:57.747730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 464/923] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:56:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:57.749807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:57.852882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5849a15-9220-4044-b6ac-d09cbcc17865 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:57.888885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:57.889553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.889837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.890119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.890916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] RBAC: Adding query filter params (interface=public, service_id=9544caa05f504efc8ed6c88e87c3c71f, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:57.891294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:57.892668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:57.898160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 460/924] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 14:56:57 2022] GET /identity/v3/endpoints?service_id=9544caa05f504efc8ed6c88e87c3c71f&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:57.903448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:57.904314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f52bb2-448b-49d5-b646-9699763c1706 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.904808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f52bb2-448b-49d5-b646-9699763c1706 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.905282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f52bb2-448b-49d5-b646-9699763c1706 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.907378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 465/925] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:56:57.913401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc746a-0cd7-4248-b16c-cefdcd354833 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:57.913760 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc746a-0cd7-4248-b16c-cefdcd354833 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:57.914109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc746a-0cd7-4248-b16c-cefdcd354833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:57.932246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-befc746a-0cd7-4248-b16c-cefdcd354833 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:57.933514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-befc746a-0cd7-4248-b16c-cefdcd354833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:58.015695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4644f-a165-406c-96a0-4a457f864704 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.015913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 466/926] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:58.029799 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc746a-0cd7-4248-b16c-cefdcd354833 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.055940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.056875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.057185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.057465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.058157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:58.059667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:58.075667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 461/927] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:56:57 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 14:56:58.154219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc746a-0cd7-4248-b16c-cefdcd354833 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.197844 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5afe-d0d1-47a1-adee-3b302304d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.198763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5afe-d0d1-47a1-adee-3b302304d338 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.199047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5afe-d0d1-47a1-adee-3b302304d338 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.199325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5afe-d0d1-47a1-adee-3b302304d338 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.213444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5afe-d0d1-47a1-adee-3b302304d338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-400f5afe-d0d1-47a1-adee-3b302304d338 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:56:58.223588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:58.224643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:56:58.225511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.230021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 467/928] 10.222.0.71 () {56 vars in 1120 bytes} [Mon May 16 14:56:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:58.231415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.338373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4ef2-5381-4aa1-97cb-1a03e0e2b7ad None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.374260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.374772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3341228-d611-4587-9d5a-dfd8463cb512 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.374958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.375077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3341228-d611-4587-9d5a-dfd8463cb512 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.375234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.375356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3341228-d611-4587-9d5a-dfd8463cb512 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.375580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.377585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:58.377700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 468/929] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:58.377916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:58.379417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:58.382637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce942f8-294b-462c-a56e-42c87c7b61a5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.382918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce942f8-294b-462c-a56e-42c87c7b61a5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.383193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce942f8-294b-462c-a56e-42c87c7b61a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.391995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 462/930] 10.222.0.71 () {56 vars in 1115 bytes} [Mon May 16 14:56:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:58.394828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.403385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dce942f8-294b-462c-a56e-42c87c7b61a5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:58.404642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dce942f8-294b-462c-a56e-42c87c7b61a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:58.476497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 469/931] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:56:58.499464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce942f8-294b-462c-a56e-42c87c7b61a5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.505941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d96df5-601a-4fa7-b330-559fab664e48 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.551334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.552183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.552470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.552747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.568915 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:58.570696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:58.594728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:58.596446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:58.597562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:58.597562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:58.597562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:58.597562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:58.604240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 463/932] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:58.606046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.617842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce942f8-294b-462c-a56e-42c87c7b61a5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.652255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.653551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.653551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.653551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.654191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:58.655704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:58.657990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING keystone.server.flask.common [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 16 14:56:58.705531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.sql.core [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 16 14:56:58.705531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 16 14:56:58.705531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [parameters: {'id': '6073d83c8a194cc2a7c69700951170d4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 16 14:56:58.705531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80233) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6073d83c8a194cc2a7c69700951170d4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6073d83c8a194cc2a7c69700951170d4. May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 14:56:58.722896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 16 14:56:58.723829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return project_ref.to_dict() May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application yield resource May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.rollback() May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.force_reraise() May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise self.value May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application session.commit() May 16 14:56:58.725612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._prepare_impl() May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.flush() May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._flush(objects) May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application compat.raise_( May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application flush_context.execute() May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 16 14:56:58.726516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rec.execute(self) May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application _emit_insert_statements( May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application c = connection._execute_20( May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self._execute_context( May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 16 14:56:58.727432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 14:56:58.733296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [parameters: {'id': '6073d83c8a194cc2a7c69700951170d4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.734500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:58.738438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.Conflict( May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6073d83c8a194cc2a7c69700951170d4. May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 470/933] 10.222.0.71 () {60 vars in 1140 bytes} [Mon May 16 14:56:58 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:58.740682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.742267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f62196-0e0d-48fb-8086-0a5aee771ede None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.779133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.780227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.780763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.781260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.782830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:58.783344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:58.784949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:58.802288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 464/934] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:58.803868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.846447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280098-15a0-48d8-8f4a-d7f71e8718e4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.880473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.881315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.881679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.881851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.888522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:58.890264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:58.905126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:58.906134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:58.908187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:58.911472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 471/935] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:58.912379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:58.928016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d79b46-f44e-4664-83a4-f673253fd332 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:58.964870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:58.966065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:58.966551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:58.967044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:58.975298 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:58.977125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:56:58.993638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:58.994673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:56:58.995667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:59.003766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 465/936] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:56:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:56:59.005267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:59.017568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8199b-e15d-4b6f-9f82-bd8e36dfc332 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:59.051566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:59.052416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.052699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.052966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.053712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:59.054009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:59.055727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:59.075750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 472/937] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:56:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:59.121493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb9b4d-dd95-42f2-b8a2-d9559bee2a74 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:59.162731 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:59.163800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.164304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.164803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.165871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:59.166444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:59.168158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:59.191128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 466/938] 10.222.0.71 () {56 vars in 1149 bytes} [Mon May 16 14:56:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:59.196972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:59.310582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbc8b1-eb93-49f4-9190-6c9fed69a245 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:59.348040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:59.349202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.349487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.349808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.363604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:59.363604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:56:59.382048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:59.383189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:59.384337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:56:59.384337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:56:59.384337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:56:59.384337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:59.386262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 473/939] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:59.392418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:59.500221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863f907-5c07-4042-b15e-66210468b2e4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:59.536464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022267f-d4d3-4c44-9329-9115ed6861bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:59.537132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022267f-d4d3-4c44-9329-9115ed6861bf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.537417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022267f-d4d3-4c44-9329-9115ed6861bf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.537688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022267f-d4d3-4c44-9329-9115ed6861bf None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.538408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022267f-d4d3-4c44-9329-9115ed6861bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:59.538709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022267f-d4d3-4c44-9329-9115ed6861bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:59.540085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022267f-d4d3-4c44-9329-9115ed6861bf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:59.555761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 467/940] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:59.559454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:59.649711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19ec0-eb90-4180-95b9-888335991cc1 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.649991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19ec0-eb90-4180-95b9-888335991cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.650264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19ec0-eb90-4180-95b9-888335991cc1 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.651931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 468/941] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:56:59.657021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc020ba-7457-465e-8126-c7e4ac490ee6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.657588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc020ba-7457-465e-8126-c7e4ac490ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.657873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc020ba-7457-465e-8126-c7e4ac490ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.664246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bb4ae-4815-4c72-91d9-6f3271e9fd8f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:59.674974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0cc020ba-7457-465e-8126-c7e4ac490ee6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:56:59.676176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0cc020ba-7457-465e-8126-c7e4ac490ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:56:59.701096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:59.701893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.702167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.702431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.709444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:59.710881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:56:59.724816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:59.725773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:56:59.726707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:56:59.730728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 474/942] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:56:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:59.732030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:59.772359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 469/943] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:56:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:56:59.788165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc020ba-7457-465e-8126-c7e4ac490ee6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:59.849125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654466b9-eef8-4fda-865e-6dc1477d0dc7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:59.889694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:59.890417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.890819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.891111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.891992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:56:59.892487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:59.893711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:59.901338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc020ba-7457-465e-8126-c7e4ac490ee6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:56:59.911234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 475/944] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:56:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:56:59.912362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:56:59.936727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:56:59.938229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:56:59.939345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:56:59.939792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:56:59.940633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:56:59.942204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:56:59.954824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:56:59.955848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 14:56:59.956707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:56:59.959817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 470/945] 10.222.0.71 () {56 vars in 1132 bytes} [Mon May 16 14:56:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:56:59.966191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:00.029531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849410c6-05ad-48b8-9db9-dcc6f98ae0f8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:00.071868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:00.072590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.072873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.073148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/4dd72a14691c48dd95845b7b46eb8c58/roles/84e2b91dd928402ab1eb6f28811b0ab3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.076020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=4dd72a14691c48dd95845b7b46eb8c58, role_id=84e2b91dd928402ab1eb6f28811b0ab3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:00.077744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:00.088988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cc878-cba8-4589-bd2b-6b88a4b1d299 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:00.123363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:00.124432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.124974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.125443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.127265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:00.127614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:00.129048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:00.137665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 471/946] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:56:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:00.140134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:00.191835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 476/947] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:56:59 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/4dd72a14691c48dd95845b7b46eb8c58/roles/84e2b91dd928402ab1eb6f28811b0ab3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:57:00.267015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d36d1-22e2-4294-a346-df2ef8fba373 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:00.306297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f2670-935e-44ba-910f-e2e9df231687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:00.307372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f2670-935e-44ba-910f-e2e9df231687 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.307896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f2670-935e-44ba-910f-e2e9df231687 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.308383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f2670-935e-44ba-910f-e2e9df231687 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.309334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f2670-935e-44ba-910f-e2e9df231687 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:00.309839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f2670-935e-44ba-910f-e2e9df231687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:00.311520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f2670-935e-44ba-910f-e2e9df231687 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:00.323520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 472/948] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:57:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:00.325369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:00.418799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eafb14-7fc2-4c7d-8bb9-c2c0a5a795fc None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.419300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eafb14-7fc2-4c7d-8bb9-c2c0a5a795fc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.419698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eafb14-7fc2-4c7d-8bb9-c2c0a5a795fc None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.421475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 473/949] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:00.427358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3cf2-d394-468d-b622-535cf16d3def None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.428032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3cf2-d394-468d-b622-535cf16d3def None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.428032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3cf2-d394-468d-b622-535cf16d3def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.439681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5bebd-df4f-48b2-a181-3f4febbf5517 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:00.458508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-bb3d3cf2-d394-468d-b622-535cf16d3def None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:00.460344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bb3d3cf2-d394-468d-b622-535cf16d3def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:00.475432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:00.476212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.476562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.476904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.477708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:00.479133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:00.533806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 474/950] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:00.557613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 477/951] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:57:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:00.558616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d3cf2-d394-468d-b622-535cf16d3def None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:00.671396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d3cf2-d394-468d-b622-535cf16d3def None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:00.715230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:00.716082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.716361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.716630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.717287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:00.718672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:00.732795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:00.733895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:00.734763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 475/952] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:57:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:00.742137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:00.854676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ebfea-3597-4cee-94d3-89383d72fc09 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:00.893792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:00.894828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:00.895316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:00.895847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:00.896761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:00.897279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:00.898839 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:00.904382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 478/953] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:57:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:00.911048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:01.015594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10234efe-d045-4e7d-b4f0-cd3199af58ef None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:01.052285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b25474-95d0-4bc8-a2f0-064d6b59bd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:01.053353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25474-95d0-4bc8-a2f0-064d6b59bd9d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:01.053841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25474-95d0-4bc8-a2f0-064d6b59bd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:01.054342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25474-95d0-4bc8-a2f0-064d6b59bd9d None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:01.055256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25474-95d0-4bc8-a2f0-064d6b59bd9d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:01.055805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25474-95d0-4bc8-a2f0-064d6b59bd9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:01.057398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25474-95d0-4bc8-a2f0-064d6b59bd9d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:01.063074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 476/954] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:57:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:01.069350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:01.174704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd430e39-3c8c-464f-94f7-cce1feb987e1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:01.209263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398135a2-d0f5-4c75-8804-4d6617071849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:01.210293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398135a2-d0f5-4c75-8804-4d6617071849 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:01.210782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398135a2-d0f5-4c75-8804-4d6617071849 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:01.211336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398135a2-d0f5-4c75-8804-4d6617071849 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:01.212342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398135a2-d0f5-4c75-8804-4d6617071849 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:01.214035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398135a2-d0f5-4c75-8804-4d6617071849 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:01.300833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 479/955] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:57:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:01.516151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e6184-ce6e-4cd2-8000-691587856036 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:01.517073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e6184-ce6e-4cd2-8000-691587856036 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:01.517256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e6184-ce6e-4cd2-8000-691587856036 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:01.518955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 477/956] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:01.524402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12549d-9ac8-4050-80fc-7bef2d1c6b40 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:01.524695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12549d-9ac8-4050-80fc-7bef2d1c6b40 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:01.524977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12549d-9ac8-4050-80fc-7bef2d1c6b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:01.543313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1d12549d-9ac8-4050-80fc-7bef2d1c6b40 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:01.545145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d12549d-9ac8-4050-80fc-7bef2d1c6b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:01.618283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 480/957] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:01.644026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e6184-ce6e-4cd2-8000-691587856036 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:01.747189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e6184-ce6e-4cd2-8000-691587856036 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:01.782855 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:01.784069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:01.784567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:01.785045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:01.785955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:01.787587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:01.789042 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING keystone.server.flask.common [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 16 14:57:01.834665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.sql.core [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 16 14:57:01.834665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 16 14:57:01.834665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [parameters: {'user_id': '279d2e05eed441b49a27b41db42b9a1e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 16 14:57:01.834665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80232) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 16 14:57:01.843107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 14:57:01.850187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application yield resource May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.rollback() May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.force_reraise() May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise self.value May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application session.commit() May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 16 14:57:01.851882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._prepare_impl() May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.flush() May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._flush(objects) May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application compat.raise_( May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application flush_context.execute() May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rec.execute(self) May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 16 14:57:01.853119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application _emit_insert_statements( May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = connection._execute_20( May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ret = self._execute_context( May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 14:57:01.854222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 14:57:01.855256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [parameters: {'user_id': '279d2e05eed441b49a27b41db42b9a1e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:01.856305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 16 14:57:01.857235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 16 14:57:01.858359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 16 14:57:01.858359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 16 14:57:01.858359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 16 14:57:01.858359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 16 14:57:01.858359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 16 14:57:01.858359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:01.858359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 478/958] 10.222.0.71 () {60 vars in 1131 bytes} [Mon May 16 14:57:01 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:01.864950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12549d-9ac8-4050-80fc-7bef2d1c6b40 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:01.968407 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12549d-9ac8-4050-80fc-7bef2d1c6b40 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:02.003761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4108b-d5f1-422c-ad27-5f0b24567381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:02.004831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4108b-d5f1-422c-ad27-5f0b24567381 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.005275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4108b-d5f1-422c-ad27-5f0b24567381 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.005695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4108b-d5f1-422c-ad27-5f0b24567381 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.013161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4108b-d5f1-422c-ad27-5f0b24567381 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:02.014782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4108b-d5f1-422c-ad27-5f0b24567381 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4ef4108b-d5f1-422c-ad27-5f0b24567381 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:57:02.030133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:02.031291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 14:57:02.033091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:02.033835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 481/959] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:57:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:02.040060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:02.142121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f56acc-53a6-4641-b9b7-5617dc36fa1d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:02.176931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5095a1-dfd7-42dc-bc3a-c0ed2e5898a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:02.177979 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5095a1-dfd7-42dc-bc3a-c0ed2e5898a8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.178459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5095a1-dfd7-42dc-bc3a-c0ed2e5898a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.178928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5095a1-dfd7-42dc-bc3a-c0ed2e5898a8 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.179960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5095a1-dfd7-42dc-bc3a-c0ed2e5898a8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:02.180480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5095a1-dfd7-42dc-bc3a-c0ed2e5898a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:02.183107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5095a1-dfd7-42dc-bc3a-c0ed2e5898a8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:02.227953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 479/960] 10.222.0.71 () {56 vars in 1101 bytes} [Mon May 16 14:57:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:02.560758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f3b66-8250-4ade-834e-6646e7d0d498 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.561110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f3b66-8250-4ade-834e-6646e7d0d498 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.561315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f3b66-8250-4ade-834e-6646e7d0d498 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.563317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 482/961] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:02.568673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511eb0ad-b4e3-4ee4-a613-f1c97c71c9b9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.568960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511eb0ad-b4e3-4ee4-a613-f1c97c71c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.569230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511eb0ad-b4e3-4ee4-a613-f1c97c71c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.585877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-511eb0ad-b4e3-4ee4-a613-f1c97c71c9b9 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:02.587364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-511eb0ad-b4e3-4ee4-a613-f1c97c71c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:02.651268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 480/962] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:02.675169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f3b66-8250-4ade-834e-6646e7d0d498 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:02.773818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f3b66-8250-4ade-834e-6646e7d0d498 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:02.805954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8f1be-4de3-434e-a3d5-4e8441b9cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:02.806759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8f1be-4de3-434e-a3d5-4e8441b9cddd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.807037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8f1be-4de3-434e-a3d5-4e8441b9cddd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.807309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8f1be-4de3-434e-a3d5-4e8441b9cddd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.817172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8f1be-4de3-434e-a3d5-4e8441b9cddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3ff8f1be-4de3-434e-a3d5-4e8441b9cddd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:02.825264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:02.826212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:02.827079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 483/963] 10.222.0.71 () {56 vars in 1120 bytes} [Mon May 16 14:57:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:02.834330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511eb0ad-b4e3-4ee4-a613-f1c97c71c9b9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:02.859907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3320160-14c9-45e1-80a2-103948524698 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.860822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3320160-14c9-45e1-80a2-103948524698 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.861305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3320160-14c9-45e1-80a2-103948524698 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.862895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 484/964] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:02.867715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f67209-d5df-4721-a6e0-1fe721bef610 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.868243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f67209-d5df-4721-a6e0-1fe721bef610 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.868243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f67209-d5df-4721-a6e0-1fe721bef610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.887395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-50f67209-d5df-4721-a6e0-1fe721bef610 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:02.889103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-50f67209-d5df-4721-a6e0-1fe721bef610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:02.945853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511eb0ad-b4e3-4ee4-a613-f1c97c71c9b9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:02.957698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 485/965] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:02.981588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f67209-d5df-4721-a6e0-1fe721bef610 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:02.986964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:02.987659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:02.987950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:02.988223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:02.988976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:02.989281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:02.990737 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:02.998997 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 481/966] 10.222.0.71 () {56 vars in 1115 bytes} [Mon May 16 14:57:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:03.005115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.095834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f67209-d5df-4721-a6e0-1fe721bef610 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.102807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ef446-0d95-4974-99e0-8f3be183fcd5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.130346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.131148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.131431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.131731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.132374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.133870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.135641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.136450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.136705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.136977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:03.146521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:03.147879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:03.148784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:03.150031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.150127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 486/967] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:57:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:03.150400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.154335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:03.167844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:03.168810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:03.169801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:03.169801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:03.169801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:03.169801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:03.169801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 482/968] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:03.174505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.257163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b662-18ab-4110-9de2-8756fc81490b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.271980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6d838-a4b5-4960-a050-8e9000c2e63f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.289667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.290325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.290605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.290875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.291570 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:03.291903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.293248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.302633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 487/969] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:03.303740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.309726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.310384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.310659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.310927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.311668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:03.311970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.313870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.325168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 483/970] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:03.330508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.404395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460cc55-8835-4bf4-ac14-0d823c6034a1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.428828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155244f-c325-403e-a691-085de8d2dd54 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.442349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.443331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.443825 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.444103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.444827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] RBAC: Adding query filter params (type=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:03.445133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.446535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.451486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 488/971] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:03.465113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003f09d-4096-49da-9602-497260c18258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.465947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003f09d-4096-49da-9602-497260c18258 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.466225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003f09d-4096-49da-9602-497260c18258 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.466494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003f09d-4096-49da-9602-497260c18258 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.476261 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003f09d-4096-49da-9602-497260c18258 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.477854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003f09d-4096-49da-9602-497260c18258 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c003f09d-4096-49da-9602-497260c18258 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:57:03.494488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:03.495546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:03.502541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 484/972] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:03.503497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.606893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18ca12-3f08-423f-bc57-be7e724deada None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.619657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692cde9-3822-49d0-8483-939c0ec0c422 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.619968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692cde9-3822-49d0-8483-939c0ec0c422 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.620210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692cde9-3822-49d0-8483-939c0ec0c422 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.621727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 485/973] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:57:03.627118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b41d5-925e-4384-91f5-31c07296a8f2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.627404 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b41d5-925e-4384-91f5-31c07296a8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.627820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b41d5-925e-4384-91f5-31c07296a8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.647289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c48b41d5-925e-4384-91f5-31c07296a8f2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:03.648543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c48b41d5-925e-4384-91f5-31c07296a8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:03.649679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f229add-322a-4676-9825-68004ad76ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.650363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f229add-322a-4676-9825-68004ad76ead None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.650650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f229add-322a-4676-9825-68004ad76ead None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.650929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f229add-322a-4676-9825-68004ad76ead None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.651803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f229add-322a-4676-9825-68004ad76ead None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:03.652119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f229add-322a-4676-9825-68004ad76ead None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.653557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f229add-322a-4676-9825-68004ad76ead None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.671875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 489/974] 10.222.0.71 () {56 vars in 1149 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:03.677699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f229add-322a-4676-9825-68004ad76ead None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.721307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 486/975] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:03.744410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b41d5-925e-4384-91f5-31c07296a8f2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:03.783346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f229add-322a-4676-9825-68004ad76ead None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.817308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.817968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.818253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.818538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.839953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b41d5-925e-4384-91f5-31c07296a8f2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:03.879797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:03.880655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:03.880992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:03.881270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:03.881917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.883269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.892506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:03.893953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:03.895514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:03.896709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:03.897573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 487/976] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:03.902549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.002782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06347a-d2d6-477a-a0af-35cfc3fea408 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.012088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 490/977] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:04.013445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.037108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.037770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.038053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.038329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.039033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:04.039336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.040732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.045752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 488/978] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:57:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:04.051872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.114721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff0bd9-2a94-4639-84c1-63bd8578c3fb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.148805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.149598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.149881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.150156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.150996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d56fe-dde2-4e6b-85ec-415fb2cdc615 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.161009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.163052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.183682 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.184343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.184703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:04.184827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:04.185654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:04.191709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:04.191709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:04.191709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:04.191709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:04.191942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.191942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:04.191942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.192157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 491/979] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:04.192229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.192298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.193042 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 489/980] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:04.198714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.290408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b65c70-cd42-4f10-aad1-6b42fdd11981 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.305106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77478-c8da-429a-b0ca-c62fe31f3dd7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.325254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.325922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.333100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.333100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.333100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:04.333100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.333100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.339038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.339716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.340003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.340275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.340716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 492/981] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:04.341149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] RBAC: Adding query filter params (interface=admin, service_id=9544caa05f504efc8ed6c88e87c3c71f, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:04.341460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.342857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.346915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.353449 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 490/982] 10.222.0.71 () {56 vars in 1249 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/endpoints?service_id=9544caa05f504efc8ed6c88e87c3c71f&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:04.354341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.442493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d1287-67c5-47c5-b96c-04bd91a70348 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.453912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e4287-2f1f-49fb-a945-b9196d1f50c6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.475586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.476406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.476683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.476951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.483318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.484805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.488292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.488954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.489238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.489513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.490222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.491581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.504660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 491/983] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/services => generated 1497 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:57:04.504887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:04.505905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:04.506881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 493/984] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:04.507383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.613253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1282268-c3e6-409d-9827-4bc2e2b8c511 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.649490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626abbe-69f6-44ac-9c01-a7c987a3e848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.650166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626abbe-69f6-44ac-9c01-a7c987a3e848 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.650451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626abbe-69f6-44ac-9c01-a7c987a3e848 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.650728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626abbe-69f6-44ac-9c01-a7c987a3e848 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.651538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626abbe-69f6-44ac-9c01-a7c987a3e848 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:04.651882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626abbe-69f6-44ac-9c01-a7c987a3e848 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.653421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626abbe-69f6-44ac-9c01-a7c987a3e848 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.667830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 492/985] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:04.672555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:04.771047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e751cd-cddc-4893-a94d-f8ea99a9f8fa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:04.802713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b012ade-ea0e-4e04-aa2d-901e312b07e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:04.803364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b012ade-ea0e-4e04-aa2d-901e312b07e8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:04.803651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b012ade-ea0e-4e04-aa2d-901e312b07e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:04.803939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b012ade-ea0e-4e04-aa2d-901e312b07e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:04.804757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b012ade-ea0e-4e04-aa2d-901e312b07e8 None admin] RBAC: Adding query filter params (role.id=84e2b91dd928402ab1eb6f28811b0ab3, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=4dd72a14691c48dd95845b7b46eb8c58) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:04.805062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b012ade-ea0e-4e04-aa2d-901e312b07e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:04.806479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b012ade-ea0e-4e04-aa2d-901e312b07e8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:04.812531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 494/986] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:04 2022] GET /identity/v3/role_assignments?user.id=4dd72a14691c48dd95845b7b46eb8c58&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=84e2b91dd928402ab1eb6f28811b0ab3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:05.699174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c5832-7a9f-4624-acc2-653d607e57e8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:05.699761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c5832-7a9f-4624-acc2-653d607e57e8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:05.699761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c5832-7a9f-4624-acc2-653d607e57e8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:05.701279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 493/987] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:05.706909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1baf6e-b19f-44d5-b3f2-a7ab5080fc9b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:05.707293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1baf6e-b19f-44d5-b3f2-a7ab5080fc9b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:05.707474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1baf6e-b19f-44d5-b3f2-a7ab5080fc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:05.724643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3b1baf6e-b19f-44d5-b3f2-a7ab5080fc9b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:05.725828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3b1baf6e-b19f-44d5-b3f2-a7ab5080fc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:05.789285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 495/988] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:05.811170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c5832-7a9f-4624-acc2-653d607e57e8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:05.915176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c5832-7a9f-4624-acc2-653d607e57e8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:05.948223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68a49e-b48f-47d5-b6c1-68d2f95ed47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:05.948882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a49e-b48f-47d5-b6c1-68d2f95ed47e None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:05.949163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a49e-b48f-47d5-b6c1-68d2f95ed47e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:05.949492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a49e-b48f-47d5-b6c1-68d2f95ed47e None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:05.950168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68a49e-b48f-47d5-b6c1-68d2f95ed47e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:05.951525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68a49e-b48f-47d5-b6c1-68d2f95ed47e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:06.055536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 494/989] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:57:05 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:06.723018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45846a2-eae2-4d01-87da-13c003ee6c69 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:06.723590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45846a2-eae2-4d01-87da-13c003ee6c69 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:06.723590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45846a2-eae2-4d01-87da-13c003ee6c69 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:06.725089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 496/990] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:06.730238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85776f66-1aeb-4e72-bf07-afd46093fe52 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:06.730634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85776f66-1aeb-4e72-bf07-afd46093fe52 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:06.730816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85776f66-1aeb-4e72-bf07-afd46093fe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:06.747515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-85776f66-1aeb-4e72-bf07-afd46093fe52 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:06.748714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-85776f66-1aeb-4e72-bf07-afd46093fe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:06.819130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 495/991] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:06.841353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45846a2-eae2-4d01-87da-13c003ee6c69 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:06.955493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45846a2-eae2-4d01-87da-13c003ee6c69 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:06.990113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3add26-b7e5-4abd-a3fc-499ca389ea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:06.990856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3add26-b7e5-4abd-a3fc-499ca389ea02 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:06.991137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3add26-b7e5-4abd-a3fc-499ca389ea02 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:06.991412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3add26-b7e5-4abd-a3fc-499ca389ea02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:06.992106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3add26-b7e5-4abd-a3fc-499ca389ea02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:06.993471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3add26-b7e5-4abd-a3fc-499ca389ea02 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4e3add26-b7e5-4abd-a3fc-499ca389ea02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:07.006227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:07.007461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:07.008330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 497/992] 10.222.0.71 () {56 vars in 1126 bytes} [Mon May 16 14:57:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:07.015429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85776f66-1aeb-4e72-bf07-afd46093fe52 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:07.102857 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4ec63-3ab2-4051-b88c-fcae62626797 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.103182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4ec63-3ab2-4051-b88c-fcae62626797 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.103418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4ec63-3ab2-4051-b88c-fcae62626797 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.106109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 498/993] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:57:07.109328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85776f66-1aeb-4e72-bf07-afd46093fe52 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:07.110052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846bd21-809e-470b-a0ac-1d4c5bd2e9cb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.110344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846bd21-809e-470b-a0ac-1d4c5bd2e9cb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.110662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846bd21-809e-470b-a0ac-1d4c5bd2e9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.127373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6846bd21-809e-470b-a0ac-1d4c5bd2e9cb None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:07.128569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6846bd21-809e-470b-a0ac-1d4c5bd2e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:07.145406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:07.146075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.146358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.146635 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.147343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:07.147668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:07.149030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:07.153823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 496/994] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:57:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:07.159356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:07.199205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 499/995] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:07.221786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846bd21-809e-470b-a0ac-1d4c5bd2e9cb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:07.258192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175c2ac-14b5-4b9d-8d62-febaf65e50cd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:07.290464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:07.291123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.291401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.291764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.292469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:07.292766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:07.294122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:07.304686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 497/996] 10.222.0.71 () {56 vars in 1122 bytes} [Mon May 16 14:57:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:07.305385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:07.318790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846bd21-809e-470b-a0ac-1d4c5bd2e9cb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:07.354403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:07.355194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.355471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.355783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] PATH_INFO: `/v3/services/image` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.356444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:07.357856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:07.371241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:07.373111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:07.373976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 500/997] 10.222.0.71 () {56 vars in 1105 bytes} [Mon May 16 14:57:07 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:07.379797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:07.409862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e252-4d0d-4431-9cbe-538da3c8201b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:07.443567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:07.444261 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.444546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.444826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.445563 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:07.446955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:07.478925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fd33e-cefd-4f0d-bbce-7c2e253f1b4e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:07.512366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f168a-df5b-4556-af26-e421d8e9c105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:07.512906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f168a-df5b-4556-af26-e421d8e9c105 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.513188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f168a-df5b-4556-af26-e421d8e9c105 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.513510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f168a-df5b-4556-af26-e421d8e9c105 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.514171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f168a-df5b-4556-af26-e421d8e9c105 None admin] RBAC: Adding query filter params (name=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:07.514466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f168a-df5b-4556-af26-e421d8e9c105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:07.515116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 498/998] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:57:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:07.516186 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f168a-df5b-4556-af26-e421d8e9c105 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:07.526321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 501/999] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:07.527363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:07.643001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f18d70-6f28-44cb-be92-b4964355c1c5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:07.679623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74e867-9486-4c38-a8bd-0d3b08e9f018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:07.680389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74e867-9486-4c38-a8bd-0d3b08e9f018 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:07.680673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74e867-9486-4c38-a8bd-0d3b08e9f018 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:07.680998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74e867-9486-4c38-a8bd-0d3b08e9f018 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:07.681727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74e867-9486-4c38-a8bd-0d3b08e9f018 None admin] RBAC: Adding query filter params (type=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:07.682031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74e867-9486-4c38-a8bd-0d3b08e9f018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:07.683498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74e867-9486-4c38-a8bd-0d3b08e9f018 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:07.695673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 499/1000] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:07 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:08.309699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff310-31ed-4340-aede-b347e9341a0e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:08.310253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff310-31ed-4340-aede-b347e9341a0e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:08.310253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff310-31ed-4340-aede-b347e9341a0e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:08.311772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 502/1001] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:08.317380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34c5c-04b4-4056-bbf0-00e20dcbcc10 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:08.317803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34c5c-04b4-4056-bbf0-00e20dcbcc10 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:08.317948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34c5c-04b4-4056-bbf0-00e20dcbcc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:08.334599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-21b34c5c-04b4-4056-bbf0-00e20dcbcc10 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:08.335822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-21b34c5c-04b4-4056-bbf0-00e20dcbcc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:08.402099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 500/1002] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:08 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:08.425250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff310-31ed-4340-aede-b347e9341a0e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:08.525006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff310-31ed-4340-aede-b347e9341a0e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:08.556969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:08.557755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:08.558036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:08.558309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:08.558951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:08.560385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:08.572459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:08.573460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:08.574324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:08.575963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 503/1003] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:57:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:08.580188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34c5c-04b4-4056-bbf0-00e20dcbcc10 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:08.676617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34c5c-04b4-4056-bbf0-00e20dcbcc10 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:08.709406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:08.710072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:08.710356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:08.710632 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:08.711335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:08.711663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:08.713030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:08.719849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 501/1004] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:57:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:08.724019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:08.823104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555883a3-8ba0-4025-aba7-54199098d90b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:08.857483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:08.858160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:08.858443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:08.858722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:08.859431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:08.859771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:08.861749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:08.866559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 504/1005] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:57:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:08.872393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:08.969851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464346-a2ac-497b-a402-6c1ef4a51489 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:09.002368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3845150-b801-4166-b021-126c670f6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:09.003020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3845150-b801-4166-b021-126c670f6369 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:09.003305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3845150-b801-4166-b021-126c670f6369 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:09.003583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3845150-b801-4166-b021-126c670f6369 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:09.004397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3845150-b801-4166-b021-126c670f6369 None admin] RBAC: Adding query filter params (interface=public, service_id=08398f0caf844c51b474a61507c8eb0b, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:09.004709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3845150-b801-4166-b021-126c670f6369 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:09.006080 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3845150-b801-4166-b021-126c670f6369 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:09.011079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 502/1006] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 14:57:08 2022] GET /identity/v3/endpoints?service_id=08398f0caf844c51b474a61507c8eb0b&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:09.016385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:09.118809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3b3b-cc55-44b0-9bc9-2f52842003f1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:09.149940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55de9a-e920-469d-a54c-a8fe34b123c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:09.150601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55de9a-e920-469d-a54c-a8fe34b123c0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:09.150878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55de9a-e920-469d-a54c-a8fe34b123c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:09.151150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55de9a-e920-469d-a54c-a8fe34b123c0 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:09.151862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55de9a-e920-469d-a54c-a8fe34b123c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:09.153210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55de9a-e920-469d-a54c-a8fe34b123c0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:09.167677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 505/1007] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:57:09 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 14:57:09.775265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334603-1eec-4127-89b8-3ffbc1054b83 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:09.775835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334603-1eec-4127-89b8-3ffbc1054b83 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:09.775835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77334603-1eec-4127-89b8-3ffbc1054b83 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:09.777342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 503/1008] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:09.782342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705124-3801-4972-a2d4-de57e63db8bf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:09.782745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705124-3801-4972-a2d4-de57e63db8bf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:09.782953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705124-3801-4972-a2d4-de57e63db8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:09.799879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cd705124-3801-4972-a2d4-de57e63db8bf None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:09.801057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cd705124-3801-4972-a2d4-de57e63db8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:09.873257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 506/1009] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:09.891733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334603-1eec-4127-89b8-3ffbc1054b83 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:09.903828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf23d7-ce1a-4c5c-bb0b-c7f9c9b706ff None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:09.904359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf23d7-ce1a-4c5c-bb0b-c7f9c9b706ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:09.904359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf23d7-ce1a-4c5c-bb0b-c7f9c9b706ff None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:09.905866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 507/1010] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:57:09.911363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ae26-aedd-4837-87b7-27450e14a1fa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:09.911669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ae26-aedd-4837-87b7-27450e14a1fa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:09.911952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ae26-aedd-4837-87b7-27450e14a1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:09.928712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1031ae26-aedd-4837-87b7-27450e14a1fa None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:09.929883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1031ae26-aedd-4837-87b7-27450e14a1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:09.996483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 508/1011] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:10.002885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77334603-1eec-4127-89b8-3ffbc1054b83 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:10.018904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031ae26-aedd-4837-87b7-27450e14a1fa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:10.036800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:10.037524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:10.037802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:10.038074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:10.039251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:10.040714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:10.042473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 504/1012] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:10.047900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:10.116712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031ae26-aedd-4837-87b7-27450e14a1fa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:10.144697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7898cd-da7a-4a23-b3b9-3a7e97ee81b4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:10.149484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0cbc3-5bbf-4cdf-a7d9-6a5cabc2782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:10.150150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0cbc3-5bbf-4cdf-a7d9-6a5cabc2782f None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:10.150433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0cbc3-5bbf-4cdf-a7d9-6a5cabc2782f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:10.150709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0cbc3-5bbf-4cdf-a7d9-6a5cabc2782f None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:10.151424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0cbc3-5bbf-4cdf-a7d9-6a5cabc2782f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:10.152826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0cbc3-5bbf-4cdf-a7d9-6a5cabc2782f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:10.177932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b387c14-e0a7-4e7a-bcc7-1a5a7c99782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:10.181326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b387c14-e0a7-4e7a-bcc7-1a5a7c99782d None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:10.181326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b387c14-e0a7-4e7a-bcc7-1a5a7c99782d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:10.181326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b387c14-e0a7-4e7a-bcc7-1a5a7c99782d None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:10.181326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b387c14-e0a7-4e7a-bcc7-1a5a7c99782d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:10.181326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b387c14-e0a7-4e7a-bcc7-1a5a7c99782d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:10.232082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 509/1013] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:57:10 2022] POST /identity/v3/services => generated 242 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 14:57:10.260630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 505/1014] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:57:10 2022] POST /identity/v3/projects => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:11.422109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0cc8f-79cf-43f1-b112-106f4f621b9a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:11.422686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0cc8f-79cf-43f1-b112-106f4f621b9a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:11.422686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0cc8f-79cf-43f1-b112-106f4f621b9a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:11.425201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 510/1015] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:11.431145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b2676-adf1-463b-87b6-328697b6e912 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:11.431723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b2676-adf1-463b-87b6-328697b6e912 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:11.431818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b2676-adf1-463b-87b6-328697b6e912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:11.448527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-680b2676-adf1-463b-87b6-328697b6e912 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:11.449718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-680b2676-adf1-463b-87b6-328697b6e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:11.520695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 506/1016] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:11 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:11.543303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0cc8f-79cf-43f1-b112-106f4f621b9a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:11.639581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0cc8f-79cf-43f1-b112-106f4f621b9a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:11.671348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:11.672221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:11.672446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:11.672719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:11.673457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:11.674820 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:11.686024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:11.686994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 14:57:11.688208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:11.691707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 511/1017] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:57:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:11.693617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b2676-adf1-463b-87b6-328697b6e912 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:11.786506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b2676-adf1-463b-87b6-328697b6e912 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:11.818175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:11.818833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:11.819111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:11.819379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:11.820107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:11.820407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:11.821757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:11.831606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 507/1018] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:57:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:11.832855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:11.938694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59ebc6-aba5-404e-9f04-44b1cac06237 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:11.972941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c2ed9-4842-4dca-942e-a8a5e46c9e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:11.973981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c2ed9-4842-4dca-942e-a8a5e46c9e61 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:11.974470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c2ed9-4842-4dca-942e-a8a5e46c9e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:11.974969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c2ed9-4842-4dca-942e-a8a5e46c9e61 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:11.975918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c2ed9-4842-4dca-942e-a8a5e46c9e61 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:11.976449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c2ed9-4842-4dca-942e-a8a5e46c9e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:11.978033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c2ed9-4842-4dca-942e-a8a5e46c9e61 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:11.983404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 512/1019] 10.222.0.71 () {56 vars in 1112 bytes} [Mon May 16 14:57:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:11.988757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:12.085510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb73b2-1a84-4c08-a1d3-455930e1523e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:12.118144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07315c4-fd5d-4bf8-bf94-aa1cb86dd091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:12.118802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07315c4-fd5d-4bf8-bf94-aa1cb86dd091 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.119081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07315c4-fd5d-4bf8-bf94-aa1cb86dd091 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.119354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07315c4-fd5d-4bf8-bf94-aa1cb86dd091 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.120135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07315c4-fd5d-4bf8-bf94-aa1cb86dd091 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:12.121490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07315c4-fd5d-4bf8-bf94-aa1cb86dd091 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:12.230320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 508/1020] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:57:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:12.492654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121c94-08d1-49c8-b851-b343a2e0989b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.493148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121c94-08d1-49c8-b851-b343a2e0989b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.493148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121c94-08d1-49c8-b851-b343a2e0989b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.494474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 513/1021] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:12.499982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f49a-fe8e-49e4-85d5-31e2f1376b2b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.500392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f49a-fe8e-49e4-85d5-31e2f1376b2b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.500565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f49a-fe8e-49e4-85d5-31e2f1376b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.518960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7507f49a-fe8e-49e4-85d5-31e2f1376b2b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:12.520213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7507f49a-fe8e-49e4-85d5-31e2f1376b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:12.552819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b4cf-ba0e-409e-9690-5deea4fc0fbd None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.553433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b4cf-ba0e-409e-9690-5deea4fc0fbd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.553910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454b4cf-ba0e-409e-9690-5deea4fc0fbd None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.555554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 514/1022] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:12.560972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e162-ec45-45c6-a50f-345a605fdfaa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.561251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e162-ec45-45c6-a50f-345a605fdfaa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.561520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e162-ec45-45c6-a50f-345a605fdfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.581409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0f88e162-ec45-45c6-a50f-345a605fdfaa None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:12.584445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0f88e162-ec45-45c6-a50f-345a605fdfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:12.596829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 509/1023] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:12.620789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f49a-fe8e-49e4-85d5-31e2f1376b2b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:12.652541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 515/1024] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:12.677113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88e162-ec45-45c6-a50f-345a605fdfaa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:12.722752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f49a-fe8e-49e4-85d5-31e2f1376b2b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:12.757809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:12.758519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.758805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.759081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.759773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:12.761149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:12.774027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:12.775070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:12.775986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:12.776756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88e162-ec45-45c6-a50f-345a605fdfaa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:12.780549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 510/1025] 10.222.0.71 () {56 vars in 1105 bytes} [Mon May 16 14:57:12 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:12.781367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:12.810367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:12.811148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.811428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.811736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.812904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:12.815084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:12.816745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 516/1026] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:12.822242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:12.890744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b5706-7d01-43b6-a2d2-f587ddcc36e7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:12.924693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:12.925358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.925636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.925899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.925996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf552f0-041e-4613-bd47-73576999ab97 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:12.926603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] RBAC: Adding query filter params (name=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:12.926902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:12.928300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:12.933072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 511/1027] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:12.938570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:12.963184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:12.963884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:12.964162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:12.964431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:12.965120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:12.966532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:13.032345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaeed3-4a00-402a-97bc-8b323aec7e18 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:13.059852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 517/1028] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:57:12 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 14:57:13.069248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:13.069939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:13.070221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:13.070491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:13.071221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] RBAC: Adding query filter params (type=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:13.071525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:13.072975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:13.078861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 512/1029] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:13.085367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:13.191452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063636fd-015a-43c7-aea1-3f97895f3cb6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:13.226539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61d14-44fc-42c8-a469-242108a8f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:13.227217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61d14-44fc-42c8-a469-242108a8f3eb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:13.227501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61d14-44fc-42c8-a469-242108a8f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:13.227809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61d14-44fc-42c8-a469-242108a8f3eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:13.228610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61d14-44fc-42c8-a469-242108a8f3eb None admin] RBAC: Adding query filter params (interface=public, service_id=aa66cfbbf14844b8a9f3913f31db39da, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:13.228920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61d14-44fc-42c8-a469-242108a8f3eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:13.230296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61d14-44fc-42c8-a469-242108a8f3eb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:13.239101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 518/1030] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 14:57:13 2022] GET /identity/v3/endpoints?service_id=aa66cfbbf14844b8a9f3913f31db39da&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:13.244327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:13.344448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b56a-6e8b-4029-a185-6c3c09e93f81 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:13.377490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc0a52-b0bb-4bc4-a7bf-7237cf703c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:13.378424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc0a52-b0bb-4bc4-a7bf-7237cf703c11 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:13.378846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc0a52-b0bb-4bc4-a7bf-7237cf703c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:13.379266 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc0a52-b0bb-4bc4-a7bf-7237cf703c11 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:13.380152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc0a52-b0bb-4bc4-a7bf-7237cf703c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:13.381652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc0a52-b0bb-4bc4-a7bf-7237cf703c11 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:13.388451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 513/1031] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 14:57:13 2022] GET /identity/v3/services => generated 1957 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 14:57:14.626722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb7f96-796f-4a37-96cd-e3b4c9cdbc5d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:14.627991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb7f96-796f-4a37-96cd-e3b4c9cdbc5d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:14.627991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb7f96-796f-4a37-96cd-e3b4c9cdbc5d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:14.628791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 519/1032] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:14.634451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7608e-2414-4c35-8a1c-9bdbf29e3987 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:14.634883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7608e-2414-4c35-8a1c-9bdbf29e3987 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:14.635079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7608e-2414-4c35-8a1c-9bdbf29e3987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:14.651615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f7b7608e-2414-4c35-8a1c-9bdbf29e3987 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:14.652820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7b7608e-2414-4c35-8a1c-9bdbf29e3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:14.716562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 514/1033] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:14.740665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb7f96-796f-4a37-96cd-e3b4c9cdbc5d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:14.842529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb7f96-796f-4a37-96cd-e3b4c9cdbc5d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:14.876474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:14.878067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:14.878303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:14.878579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:14.888999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:14.897292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:14.898357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:14.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:14.903361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 520/1034] 10.222.0.71 () {56 vars in 1120 bytes} [Mon May 16 14:57:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:14.904644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7608e-2414-4c35-8a1c-9bdbf29e3987 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.009688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7608e-2414-4c35-8a1c-9bdbf29e3987 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.042096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.042768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.043054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.043331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.044119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:15.044429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.045800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:15.058806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 515/1035] 10.222.0.71 () {56 vars in 1115 bytes} [Mon May 16 14:57:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:15.060455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.161763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efdeb5-546e-4f9a-8987-95698836d163 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.198140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b50faf-b3d8-4bfb-9138-bef25be3de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.198938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50faf-b3d8-4bfb-9138-bef25be3de2d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.199219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50faf-b3d8-4bfb-9138-bef25be3de2d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.199493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50faf-b3d8-4bfb-9138-bef25be3de2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.210065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b50faf-b3d8-4bfb-9138-bef25be3de2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.211515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b50faf-b3d8-4bfb-9138-bef25be3de2d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-61b50faf-b3d8-4bfb-9138-bef25be3de2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:15.228849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:15.229887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:15.235550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:15.235550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:15.235550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:15.235550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:15.235550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 521/1036] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:15.237367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.335040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08534632-d37f-440a-9ba3-25010b8f4d0a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.340520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2229b4-3729-4bbc-badc-4a1260881275 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.340806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2229b4-3729-4bbc-badc-4a1260881275 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.341079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2229b4-3729-4bbc-badc-4a1260881275 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.342654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 522/1037] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:57:15.347759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305e765-4472-49ae-904c-5cd9b9a4f101 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.348045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305e765-4472-49ae-904c-5cd9b9a4f101 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.348318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305e765-4472-49ae-904c-5cd9b9a4f101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.364627 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8305e765-4472-49ae-904c-5cd9b9a4f101 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:15.366189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8305e765-4472-49ae-904c-5cd9b9a4f101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:15.369744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.370390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.370653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.370926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.371678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:15.371966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.373316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:15.387887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 516/1038] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:15.393731 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.452552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 523/1039] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:15.476219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305e765-4472-49ae-904c-5cd9b9a4f101 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.510752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dde42-2822-48d4-b9f9-fcf730081850 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.542729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.543519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.543836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.544082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.551972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.553457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:57:15.568587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:15.569640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:15.571083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 517/1040] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:15.575193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.578056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305e765-4472-49ae-904c-5cd9b9a4f101 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.613389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097755c2-2e00-4694-be35-965a423eaf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.614185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097755c2-2e00-4694-be35-965a423eaf1a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.614463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097755c2-2e00-4694-be35-965a423eaf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.614731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097755c2-2e00-4694-be35-965a423eaf1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.624316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097755c2-2e00-4694-be35-965a423eaf1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-097755c2-2e00-4694-be35-965a423eaf1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:15.632517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:15.633999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:15.634868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 524/1041] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:15.635592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70dd0-3389-4d49-9a5d-25a63e99f424 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.635592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70dd0-3389-4d49-9a5d-25a63e99f424 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.636060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70dd0-3389-4d49-9a5d-25a63e99f424 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.637293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 525/1042] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:15.639392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd70dd0-3389-4d49-9a5d-25a63e99f424 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.672689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6971afb-b6ca-45da-8b4e-f20b59b1b033 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.705250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f8f4e-0c6f-4f33-acd0-563ed9e4691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.705922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f8f4e-0c6f-4f33-acd0-563ed9e4691e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.706203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f8f4e-0c6f-4f33-acd0-563ed9e4691e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.706474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f8f4e-0c6f-4f33-acd0-563ed9e4691e None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.707301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f8f4e-0c6f-4f33-acd0-563ed9e4691e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:15.707609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f8f4e-0c6f-4f33-acd0-563ed9e4691e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.709051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f8f4e-0c6f-4f33-acd0-563ed9e4691e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:15.725604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 518/1043] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:15.728006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02b884-c62a-48af-90f3-356e921a1c00 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.728363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02b884-c62a-48af-90f3-356e921a1c00 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.728716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02b884-c62a-48af-90f3-356e921a1c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.734516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd70dd0-3389-4d49-9a5d-25a63e99f424 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.744793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-bc02b884-c62a-48af-90f3-356e921a1c00 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:15.745979 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bc02b884-c62a-48af-90f3-356e921a1c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:15.767151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.768455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.768903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.769176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.769915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:15.770214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.771565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:15.779780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 526/1044] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:15.780556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.816278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 519/1045] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:15.817025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02b884-c62a-48af-90f3-356e921a1c00 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.878639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eb52d-d84f-42bd-b1d5-54178cf43029 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.911535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.912206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.912488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.912768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.915436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02b884-c62a-48af-90f3-356e921a1c00 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:15.948382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:15.949248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:15.949529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:15.949805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] PATH_INFO: `/v3/users/47f44eceef3d4b4590f875fea4516468` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:15.962459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] RBAC: Authorizing `identity:get_user(user_id=47f44eceef3d4b4590f875fea4516468)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.963854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:15.965672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 520/1046] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/users/47f44eceef3d4b4590f875fea4516468 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:15.966273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:15.982103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:15.983542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.062250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad9054-f377-4e9c-b7fa-f5f478e8d12c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.075559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 527/1047] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:16.076420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.093887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.094663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.094940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.096029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] PATH_INFO: `/v3/services/image` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.096835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.098385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:16.109509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:16.115941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:16.116886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 521/1048] 10.222.0.71 () {56 vars in 1105 bytes} [Mon May 16 14:57:15 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:16.117732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.170328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b58c-3978-452f-a196-ee365251d7c9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.203439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.204111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.204396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.204671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.209677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca62ff4-5bf0-44e4-959d-4018d3f1554c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.242033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.242749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.243034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.243313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.253385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.254830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.268282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.269719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:16.272491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:16.273384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:16.274317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:16.274317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:16.274317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:16.274317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:16.274317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 522/1049] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:16.274617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.364284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 528/1050] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:16.365122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.379692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32828ad-5da5-4fd1-aaa4-ac634727d7b7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.420691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.421346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.421628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.421903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.422604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] RBAC: Adding query filter params (name=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:16.422908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.424844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.429733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 523/1051] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:16.430507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.488214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fb9c8-14d2-40cd-9bff-d2e713a7fe2b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.521374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.521956 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.522238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.522515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.523229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:16.523533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.523913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e0ee8-effd-42e4-ba2c-5bc36606dd71 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.524916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.536117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 529/1052] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:16.536904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.556043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.556889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.557171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.557448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.565703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] RBAC: Authorizing `identity:get_project(project_id=52a0a003fc9e4f0ebf9e13f567cb8708)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.567219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.568994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 524/1053] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:16.569754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.633159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7928e-d502-4d95-8b72-c1717318de91 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.664986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.665640 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.665974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.666251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.666944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] RBAC: Adding query filter params (type=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:16.667243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.668625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.673174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 530/1054] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:16.674572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.681553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80fbc3-e60e-461a-b126-bb3c15a08d63 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.714439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.715223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.715501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.715807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.723552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.725004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:57:16.739882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:16.741004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 525/1055] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:16.742055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.771039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f643-14b5-45d1-884b-3b0536973531 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.805587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721947c0-7471-4a55-b958-380474022b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.806170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721947c0-7471-4a55-b958-380474022b38 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.806450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721947c0-7471-4a55-b958-380474022b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.806722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721947c0-7471-4a55-b958-380474022b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.807536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721947c0-7471-4a55-b958-380474022b38 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708, user.id=47f44eceef3d4b4590f875fea4516468) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:16.807873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721947c0-7471-4a55-b958-380474022b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.809288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721947c0-7471-4a55-b958-380474022b38 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.814593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 531/1056] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/role_assignments?user.id=47f44eceef3d4b4590f875fea4516468&scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:16.815401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721947c0-7471-4a55-b958-380474022b38 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.864271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31263e-136b-4e93-9da4-0ef105c7d38d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.905051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cb94b-a143-4f46-9944-7f82795f2df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.905728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cb94b-a143-4f46-9944-7f82795f2df8 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.906024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cb94b-a143-4f46-9944-7f82795f2df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.906297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cb94b-a143-4f46-9944-7f82795f2df8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.907054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cb94b-a143-4f46-9944-7f82795f2df8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.909704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cb94b-a143-4f46-9944-7f82795f2df8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.932627 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721947c0-7471-4a55-b958-380474022b38 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:16.968577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:16.969250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:16.971390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:16.971390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:16.971675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:16.971961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:16.973409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:16.987620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 532/1057] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:16.993700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:16.997423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 526/1058] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 14:57:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:17.104119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d016b7-7a7e-46a9-acdf-772be9c7f2ca None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:17.146598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158a82e-c9f4-409b-a5e1-aa549ff0d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:17.147262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158a82e-c9f4-409b-a5e1-aa549ff0d262 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:17.147543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158a82e-c9f4-409b-a5e1-aa549ff0d262 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:17.147840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158a82e-c9f4-409b-a5e1-aa549ff0d262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:17.148675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158a82e-c9f4-409b-a5e1-aa549ff0d262 None admin] RBAC: Adding query filter params (role.id=84e2b91dd928402ab1eb6f28811b0ab3, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=3d58a6e2eac047ad8e88a1f973ecc697) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:17.148980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158a82e-c9f4-409b-a5e1-aa549ff0d262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:17.150423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158a82e-c9f4-409b-a5e1-aa549ff0d262 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:17.156476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 533/1059] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:16 2022] GET /identity/v3/role_assignments?user.id=3d58a6e2eac047ad8e88a1f973ecc697&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=84e2b91dd928402ab1eb6f28811b0ab3 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:19.114022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5e0b-cefb-4f5c-94b7-e4b7ea524aae None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.114518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5e0b-cefb-4f5c-94b7-e4b7ea524aae None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.114518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5e0b-cefb-4f5c-94b7-e4b7ea524aae None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.115890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 527/1060] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:19.121187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967efe-e5f6-4244-8381-554bdd596779 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.121604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967efe-e5f6-4244-8381-554bdd596779 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.121771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967efe-e5f6-4244-8381-554bdd596779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.138651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-14967efe-e5f6-4244-8381-554bdd596779 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:19.139870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-14967efe-e5f6-4244-8381-554bdd596779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:19.212651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 534/1061] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:19.235791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de5e0b-cefb-4f5c-94b7-e4b7ea524aae None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.265308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9559-6af3-4907-88f2-444265483037 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.265595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9559-6af3-4907-88f2-444265483037 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.265874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9559-6af3-4907-88f2-444265483037 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.267383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 535/1062] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:57:19.272612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8762f3-798a-4b51-b0f4-c3c9a2936cb3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.272893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8762f3-798a-4b51-b0f4-c3c9a2936cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.273165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8762f3-798a-4b51-b0f4-c3c9a2936cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.289218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7d8762f3-798a-4b51-b0f4-c3c9a2936cb3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:19.290371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d8762f3-798a-4b51-b0f4-c3c9a2936cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:19.339007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de5e0b-cefb-4f5c-94b7-e4b7ea524aae None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:19.358598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 536/1063] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:19.367159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b356c27-a9ae-4c08-9bd8-43410f6093b7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.367390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b356c27-a9ae-4c08-9bd8-43410f6093b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.367693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b356c27-a9ae-4c08-9bd8-43410f6093b7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.369146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 537/1064] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:57:19.373686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee1689-21b7-4c43-8e27-feef66b2de0c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.373798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:19.373985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee1689-21b7-4c43-8e27-feef66b2de0c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.374245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee1689-21b7-4c43-8e27-feef66b2de0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.374545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.374823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.375139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.386284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:19.391030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-62ee1689-21b7-4c43-8e27-feef66b2de0c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:19.392226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-62ee1689-21b7-4c43-8e27-feef66b2de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:19.393954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:19.395093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 528/1065] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:19.396847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.456553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 538/1066] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:19.457382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee1689-21b7-4c43-8e27-feef66b2de0c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.490983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2ab01-f5d8-48e2-b469-9803397d3839 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:19.522700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:19.523349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.523655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.523940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.550782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee1689-21b7-4c43-8e27-feef66b2de0c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:19.583452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:19.584139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.584423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.584699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.585434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:19.585740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:19.587099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:19.587772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:19.589206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:19.594833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 539/1067] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:19.596439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.694628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 529/1068] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:19.694628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.694895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde187d-d47a-4cee-a535-a01e38623222 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:19.723149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:19.723981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.724266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.724541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.734184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:19.742453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:19.743409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 540/1069] 10.222.0.71 () {56 vars in 1120 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:19.744592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.785010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80221c-2935-4f28-9b53-1448f605430d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:19.816655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:19.817509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.817786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.818047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] PATH_INFO: `/v3/users/47f44eceef3d4b4590f875fea4516468` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.819205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=47f44eceef3d4b4590f875fea4516468)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:19.820644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:19.822261 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 530/1070] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/users/47f44eceef3d4b4590f875fea4516468 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:19.823006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.838191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50394f-aba3-48cc-b0cb-f692c9d3555c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:19.870571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:19.875779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.875779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.875779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.883156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:19.884618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:19.901579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:19.910629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:19.911585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:19.911585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:19.911585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:19.911585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:19.911585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 541/1071] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:19.911585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.920461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692253-62d4-4af0-b247-702c29cb8cb9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:19.954039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:19.954804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:19.955175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:19.955491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:19.956431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:19.956740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:19.958174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:19.966505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 531/1072] 10.222.0.71 () {56 vars in 1115 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:19.967299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:19.999272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff19ee9-87ec-4760-9995-b5943ce8dfb3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.031124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.032000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.032283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.032557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.033755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] RBAC: Authorizing `identity:get_project(project_id=52a0a003fc9e4f0ebf9e13f567cb8708)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.035346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.037068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 542/1073] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:20.037821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:20.065952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d3951-7826-496a-91dc-c603d4d863e3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.096963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.097611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.097890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.098161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.098871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:20.099171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.100550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.112104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 532/1074] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:20.112862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:20.134428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad959d-bdf5-4998-baed-8bb9a0fe5a1b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.170525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.171339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.171616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.171988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.184398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.185930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:20.204758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:20.205801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:20.206731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:20.206731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:20.206731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:20.206731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:20.206731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 543/1075] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:20.212181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:20.217122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011fa85-f31e-4682-bc78-0fc349b8c005 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.252821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.253836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.254120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.254395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708/users/47f44eceef3d4b4590f875fea4516468/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.256475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=52a0a003fc9e4f0ebf9e13f567cb8708, user_id=47f44eceef3d4b4590f875fea4516468, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.257926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.313330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdb251-c1a2-4be1-88f3-960efa00f6dd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.349292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.349954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.350236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.350512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.351241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:20.351580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.352958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.365902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 544/1076] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:20.371386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:20.399371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 533/1077] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:57:20 2022] PUT /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708/users/47f44eceef3d4b4590f875fea4516468/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:57:20.477166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d20c-a80a-4561-b641-3c5e92f864f2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.518566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.519521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.519889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.520171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.528442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.529913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:57:20.545814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:20.547087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:20.553895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 545/1078] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:57:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:20.554553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:20.651782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069d0a1-a407-4f05-8c53-09cb569903f1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.684161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.684819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.685096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.685364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.686193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:20.686496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.687944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.704525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 534/1079] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:57:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:20.710487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:20.805185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1083a7f-6b57-4fc8-bf21-6938f35b6eaa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:20.836864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:20.837645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:20.837924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:20.838197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:20.848891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:20.850330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:20.867327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:20.868439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:20.869363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:20.869363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:20.869363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:20.869363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:20.869363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 546/1080] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:20.874314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:20.968905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a008270-abd2-4604-a673-7b3ac19f3d6d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:21.001781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:21.002657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:21.002939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:21.003208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:21.003957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:21.004257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:21.005617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:21.016468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 535/1081] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:21.022352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:21.119168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d02d2-ed36-4277-84f3-4c3176c6ff83 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:21.150961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:21.151789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:21.152072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:21.152345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:21.159572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:21.161031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:57:21.174959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:21.176126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:21.177150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 547/1082] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:57:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:21.182020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:21.276397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04f8e5-4b19-47ac-9bba-1341481f3c06 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:21.307942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5609ec-94b1-4f4b-b1f1-f0d2e736cb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:21.308587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5609ec-94b1-4f4b-b1f1-f0d2e736cb02 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:21.308863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5609ec-94b1-4f4b-b1f1-f0d2e736cb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:21.309132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5609ec-94b1-4f4b-b1f1-f0d2e736cb02 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:21.309900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5609ec-94b1-4f4b-b1f1-f0d2e736cb02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:21.310207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5609ec-94b1-4f4b-b1f1-f0d2e736cb02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:21.311604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5609ec-94b1-4f4b-b1f1-f0d2e736cb02 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:21.322962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 536/1083] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:57:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:21.328862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:21.423088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa1dbd-24f2-4256-b501-62f909d1db98 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:21.454093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fa598-498c-40ae-a566-8e3acc6a0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:21.454964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fa598-498c-40ae-a566-8e3acc6a0b15 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:21.455243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fa598-498c-40ae-a566-8e3acc6a0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:21.456099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fa598-498c-40ae-a566-8e3acc6a0b15 None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/3d58a6e2eac047ad8e88a1f973ecc697/roles/84e2b91dd928402ab1eb6f28811b0ab3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:21.458250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fa598-498c-40ae-a566-8e3acc6a0b15 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=3d58a6e2eac047ad8e88a1f973ecc697, role_id=84e2b91dd928402ab1eb6f28811b0ab3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:21.459697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fa598-498c-40ae-a566-8e3acc6a0b15 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:21.530033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 548/1084] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:57:21 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/3d58a6e2eac047ad8e88a1f973ecc697/roles/84e2b91dd928402ab1eb6f28811b0ab3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 14:57:22.670921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf573ad-b888-4efc-88e5-f0716a07df61 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:22.671475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf573ad-b888-4efc-88e5-f0716a07df61 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:22.671475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf573ad-b888-4efc-88e5-f0716a07df61 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:22.672905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 537/1085] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:22.678867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492065a8-74e4-4270-a5a9-97c1ffcd5009 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:22.679303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492065a8-74e4-4270-a5a9-97c1ffcd5009 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:22.679408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492065a8-74e4-4270-a5a9-97c1ffcd5009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:22.696082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-492065a8-74e4-4270-a5a9-97c1ffcd5009 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:22.697241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-492065a8-74e4-4270-a5a9-97c1ffcd5009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:22.763842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 549/1086] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:22.787070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf573ad-b888-4efc-88e5-f0716a07df61 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:22.893715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf573ad-b888-4efc-88e5-f0716a07df61 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:22.929899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:22.930611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:22.930893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:22.931167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:22.941604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:22.950522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:22.951702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:22.958313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 538/1087] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:22.960310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492065a8-74e4-4270-a5a9-97c1ffcd5009 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:23.075200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492065a8-74e4-4270-a5a9-97c1ffcd5009 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:23.123157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:23.124127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:23.124415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:23.124789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:23.126010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:23.126321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:23.128812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:23.141481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 550/1088] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:57:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:23.148453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:23.267660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698eced-6df6-45b1-a4be-1f32e546fee9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:23.310698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:23.311567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:23.311875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:23.312147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] PATH_INFO: `/v3/users/47f44eceef3d4b4590f875fea4516468` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:23.313358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] RBAC: Authorizing `identity:get_user(user_id=47f44eceef3d4b4590f875fea4516468)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:23.315344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:23.317430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 539/1089] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:23 2022] GET /identity/v3/users/47f44eceef3d4b4590f875fea4516468 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:23.323327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:23.445547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fae106-c6d5-428a-9989-2e9632e537ad None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:23.479451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e17eb0-7d62-480e-957a-229a01d9afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:23.480136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e17eb0-7d62-480e-957a-229a01d9afe9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:23.480416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e17eb0-7d62-480e-957a-229a01d9afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:23.480688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e17eb0-7d62-480e-957a-229a01d9afe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:23.550600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e17eb0-7d62-480e-957a-229a01d9afe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:23.552013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e17eb0-7d62-480e-957a-229a01d9afe9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:23.655044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 551/1090] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:23.661506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:23.759984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11769c-cf68-4eb2-b186-4dd61e084637 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:23.761338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8bd9-50e4-49dd-8ae9-82a308bab6ac None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:23.761751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8bd9-50e4-49dd-8ae9-82a308bab6ac None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:23.761895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8bd9-50e4-49dd-8ae9-82a308bab6ac None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:23.763272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 552/1091] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:57:23.768291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded2ad7-9cea-4612-b76c-2577c549ad1a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:23.768579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded2ad7-9cea-4612-b76c-2577c549ad1a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:23.768986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded2ad7-9cea-4612-b76c-2577c549ad1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:23.785216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9ded2ad7-9cea-4612-b76c-2577c549ad1a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:23.786409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9ded2ad7-9cea-4612-b76c-2577c549ad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:23.793317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:23.794184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:23.794464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:23.794735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:23.795937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=52a0a003fc9e4f0ebf9e13f567cb8708)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:23.797374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:23.799084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 540/1092] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:23 2022] GET /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:23.805615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:23.850229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 553/1093] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:23.874035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded2ad7-9cea-4612-b76c-2577c549ad1a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:23.902406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392492c4-5049-4e35-91c1-2cc4e4f42cc3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:23.934232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:23.934875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:23.935155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:23.935426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:23.936284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708, user.id=47f44eceef3d4b4590f875fea4516468) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:23.936588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:23.938028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:23.944444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 541/1094] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:23 2022] GET /identity/v3/role_assignments?user.id=47f44eceef3d4b4590f875fea4516468&scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:23.971409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded2ad7-9cea-4612-b76c-2577c549ad1a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:24.009211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:24.010057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:24.010338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:24.010609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:24.021813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:24.030129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:24.031313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:24.032242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 554/1095] 10.222.0.71 () {56 vars in 1120 bytes} [Mon May 16 14:57:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:24.038078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:24.155758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395abdc8-1b7e-479e-a946-897dce78bc13 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:24.188815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:24.189487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:24.189768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:24.190916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:24.191863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:24.192174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:24.193532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:24.201288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 542/1096] 10.222.0.71 () {56 vars in 1115 bytes} [Mon May 16 14:57:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:24.207527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:24.304425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a36ac-052f-4d4c-8931-279f8ebd415b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:24.337644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:24.338486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:24.338771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:24.339044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:24.349897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:24.351339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:24.368143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:24.369258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:24.370451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:24.370451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:24.370451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:24.370451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:24.371955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 555/1097] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:24.374902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:24.470578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8fbb7-40f4-4f5c-b83e-3e1ae064b8d6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:24.503417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:24.504106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:24.504387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:24.504661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:24.505376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:24.505681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:24.507039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:24.518948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 543/1098] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:24.524619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:24.620706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4c007-38b4-4d58-85f7-dc3e4ce7cfbd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:24.652705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:24.653494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:24.653775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:24.654049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:24.661605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:24.663035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:57:24.677434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:24.678489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:24.679882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 556/1099] 10.222.0.71 () {56 vars in 1093 bytes} [Mon May 16 14:57:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:24.684231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:24.778969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5845e94-d8cf-4770-97f9-27ef5d0fac4e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:24.811514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:24.812211 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:24.812489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:24.812763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:24.813594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:24.813900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:24.815977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:24.832928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 544/1100] 10.222.0.71 () {56 vars in 1133 bytes} [Mon May 16 14:57:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:24.838628 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:24.933418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319dda7-511f-4588-a60e-7606c890ab0c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:24.965808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:24.966448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:24.966732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:24.967010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:25.030946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:25.032364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:25.128368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 557/1101] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 14:57:25.134038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:25.230907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2bb6-71eb-4eb7-9774-731dfc20db50 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:25.263353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:25.264176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:25.264456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:25.264726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:25.274155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:25.275588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:25.292582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:25.293478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:25.294474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:57:25.294474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:25.294474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:57:25.294474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:25.294474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 545/1102] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:25.299925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:25.395805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309bc4a-3ce6-4079-af5e-8bd5d934be90 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:25.428139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:25.428791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:25.429074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:25.429348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:25.430060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:25.430363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:25.431744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:25.443237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 558/1103] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:25.448917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:25.543012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418fc19-c0f3-42d2-b11f-d74c6c490cad None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:25.576002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:25.576783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:25.577066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:25.577337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:25.583907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:25.585331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:57:25.600328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:25.601300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:25.602207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 546/1104] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:57:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:25.607351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:25.702246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15950dfd-638b-4922-943a-6efa4717e4e6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:25.734191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cc625-0905-4e35-9fb9-160e3efc9074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:25.734845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc625-0905-4e35-9fb9-160e3efc9074 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:25.735123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc625-0905-4e35-9fb9-160e3efc9074 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:25.735393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc625-0905-4e35-9fb9-160e3efc9074 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:25.736205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc625-0905-4e35-9fb9-160e3efc9074 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:25.736508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc625-0905-4e35-9fb9-160e3efc9074 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:25.738643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc625-0905-4e35-9fb9-160e3efc9074 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:25.750125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 559/1105] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:57:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:25.756174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:25.849851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1915-f882-4fc7-9b05-5d4eda552b06 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:25.881929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf9680-3a25-4640-82e9-7b8d5539937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:25.882571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9680-3a25-4640-82e9-7b8d5539937d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:25.882847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9680-3a25-4640-82e9-7b8d5539937d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:25.883114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9680-3a25-4640-82e9-7b8d5539937d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:25.883956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf9680-3a25-4640-82e9-7b8d5539937d None admin] RBAC: Adding query filter params (role.id=84e2b91dd928402ab1eb6f28811b0ab3, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=3d58a6e2eac047ad8e88a1f973ecc697) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:25.884264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf9680-3a25-4640-82e9-7b8d5539937d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:25.885670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf9680-3a25-4640-82e9-7b8d5539937d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:25.891200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 547/1106] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:25 2022] GET /identity/v3/role_assignments?user.id=3d58a6e2eac047ad8e88a1f973ecc697&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=84e2b91dd928402ab1eb6f28811b0ab3 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:26.208742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbd57c-04d0-4df6-9406-58514d1d5cb7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:26.209261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbd57c-04d0-4df6-9406-58514d1d5cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:26.209357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbd57c-04d0-4df6-9406-58514d1d5cb7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:26.210664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 560/1107] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:26.216204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887ce2-b968-426a-9504-4fb38f85c595 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:26.216494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887ce2-b968-426a-9504-4fb38f85c595 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:26.216767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887ce2-b968-426a-9504-4fb38f85c595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:26.233813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-91887ce2-b968-426a-9504-4fb38f85c595 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:26.234985 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-91887ce2-b968-426a-9504-4fb38f85c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:26.303015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 548/1108] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:26.328464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbd57c-04d0-4df6-9406-58514d1d5cb7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:26.431295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbd57c-04d0-4df6-9406-58514d1d5cb7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:26.467069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cecfb-6d89-4aab-91c0-97479579e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:26.467833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cecfb-6d89-4aab-91c0-97479579e767 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:26.468113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cecfb-6d89-4aab-91c0-97479579e767 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:26.468385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cecfb-6d89-4aab-91c0-97479579e767 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:26.469538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cecfb-6d89-4aab-91c0-97479579e767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:26.470968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cecfb-6d89-4aab-91c0-97479579e767 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:26.473573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 561/1109] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:26.479348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91887ce2-b968-426a-9504-4fb38f85c595 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:26.577063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91887ce2-b968-426a-9504-4fb38f85c595 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:26.611190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806ae55-8625-41a3-b7b2-29f6ffef1288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:26.611924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806ae55-8625-41a3-b7b2-29f6ffef1288 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:26.612894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806ae55-8625-41a3-b7b2-29f6ffef1288 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:26.613175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806ae55-8625-41a3-b7b2-29f6ffef1288 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:26.613900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806ae55-8625-41a3-b7b2-29f6ffef1288 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:26.615328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806ae55-8625-41a3-b7b2-29f6ffef1288 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:26.713466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 549/1110] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:57:26 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:28.966956 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804d2d2-fcd0-4261-9a69-b5a77681eff4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:28.967475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804d2d2-fcd0-4261-9a69-b5a77681eff4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:28.967475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804d2d2-fcd0-4261-9a69-b5a77681eff4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:28.968889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 562/1111] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:28.974923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8904d10-362f-41f4-a1d4-29dbd78b158d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:28.975367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8904d10-362f-41f4-a1d4-29dbd78b158d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:28.975476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8904d10-362f-41f4-a1d4-29dbd78b158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:28.992420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e8904d10-362f-41f4-a1d4-29dbd78b158d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:28.993603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e8904d10-362f-41f4-a1d4-29dbd78b158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:29.056685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 550/1112] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:29.079166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804d2d2-fcd0-4261-9a69-b5a77681eff4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:29.176939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804d2d2-fcd0-4261-9a69-b5a77681eff4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:29.209660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:29.210508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:29.210786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:29.211057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] PATH_INFO: `/v3/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:29.212543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] RBAC: Authorizing `identity:get_role(role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:29.213905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:29.215263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 563/1113] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:29 2022] GET /identity/v3/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:29.220913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8904d10-362f-41f4-a1d4-29dbd78b158d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:29.315005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8904d10-362f-41f4-a1d4-29dbd78b158d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:29.348420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:29.349267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:29.349600 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:29.349931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] PATH_INFO: `/v3/users/8fbd3678733b4165b9be465c46f4f48e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:29.363785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] RBAC: Authorizing `identity:get_user(user_id=8fbd3678733b4165b9be465c46f4f48e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:29.365206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:29.366880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 551/1114] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:29 2022] GET /identity/v3/users/8fbd3678733b4165b9be465c46f4f48e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:29.372549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:29.465169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730e2d4-7f82-44c2-ba53-a12f63a2b8a5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:29.497456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:29.498085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:29.498363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:29.498635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:29.560898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:29.562330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:29.653208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 564/1115] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:29.659593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:29.753420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc183-ce9c-4277-b781-3b0d118385ca None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:29.785181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559e441-544e-4b4c-bc47-84f1089a4845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:29.786016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559e441-544e-4b4c-bc47-84f1089a4845 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:29.786294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559e441-544e-4b4c-bc47-84f1089a4845 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:29.786565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559e441-544e-4b4c-bc47-84f1089a4845 None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:29.787747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559e441-544e-4b4c-bc47-84f1089a4845 None admin] RBAC: Authorizing `identity:get_project(project_id=52a0a003fc9e4f0ebf9e13f567cb8708)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:29.789203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559e441-544e-4b4c-bc47-84f1089a4845 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:29.790872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 552/1116] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:29 2022] GET /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:29.796909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:29.891217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169dc26-6841-4931-b036-bead0ee87185 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:29.922984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95135b23-6607-42b9-8b73-1a4f8197f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:29.923650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135b23-6607-42b9-8b73-1a4f8197f98d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:29.923951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135b23-6607-42b9-8b73-1a4f8197f98d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:29.924229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135b23-6607-42b9-8b73-1a4f8197f98d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:29.925572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95135b23-6607-42b9-8b73-1a4f8197f98d None admin] RBAC: Adding query filter params (role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708, user.id=8fbd3678733b4165b9be465c46f4f48e) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:29.925878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95135b23-6607-42b9-8b73-1a4f8197f98d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:29.927306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95135b23-6607-42b9-8b73-1a4f8197f98d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:29.932624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 565/1117] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:29 2022] GET /identity/v3/role_assignments?user.id=8fbd3678733b4165b9be465c46f4f48e&scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:32.078380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91016-1287-4de8-8471-155322e04b03 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:32.078899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91016-1287-4de8-8471-155322e04b03 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:32.078899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91016-1287-4de8-8471-155322e04b03 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:32.080305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 553/1118] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:32.086106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aab38-c628-4eed-92cb-3c49eec5b020 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:32.086523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aab38-c628-4eed-92cb-3c49eec5b020 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:32.086660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aab38-c628-4eed-92cb-3c49eec5b020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:32.103752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-096aab38-c628-4eed-92cb-3c49eec5b020 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:32.104871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-096aab38-c628-4eed-92cb-3c49eec5b020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:32.166679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 566/1119] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:32.188267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91016-1287-4de8-8471-155322e04b03 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:32.281330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91016-1287-4de8-8471-155322e04b03 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:32.312468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:32.313294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:32.313567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:32.313832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] PATH_INFO: `/v3/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:32.315280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] RBAC: Authorizing `identity:get_role(role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:32.317257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:32.318610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 554/1120] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:32 2022] GET /identity/v3/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:32.324279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aab38-c628-4eed-92cb-3c49eec5b020 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:32.418279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aab38-c628-4eed-92cb-3c49eec5b020 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:32.451301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:32.452190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:32.452472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:32.452744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] PATH_INFO: `/v3/users/8fbd3678733b4165b9be465c46f4f48e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:32.453924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] RBAC: Authorizing `identity:get_user(user_id=8fbd3678733b4165b9be465c46f4f48e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:32.455336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:32.457815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 567/1121] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:32 2022] GET /identity/v3/users/8fbd3678733b4165b9be465c46f4f48e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:32.463337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:32.556435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a3192-6cee-4afd-9732-ec2c27cfe0d1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:32.588936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b89db-ed4d-4051-8b51-36d3c22e453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:32.589773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b89db-ed4d-4051-8b51-36d3c22e453c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:32.590050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b89db-ed4d-4051-8b51-36d3c22e453c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:32.590320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b89db-ed4d-4051-8b51-36d3c22e453c None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:32.591466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b89db-ed4d-4051-8b51-36d3c22e453c None admin] RBAC: Authorizing `identity:get_project(project_id=52a0a003fc9e4f0ebf9e13f567cb8708)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:32.592902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b89db-ed4d-4051-8b51-36d3c22e453c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:32.594556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 555/1122] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:32 2022] GET /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:32.600462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:32.693282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adcf61-e6a7-4834-b51c-fa7d0763ce2c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:32.724549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4451f-5cab-4697-ae2f-3df9c9698aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:32.725406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4451f-5cab-4697-ae2f-3df9c9698aa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:32.725679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4451f-5cab-4697-ae2f-3df9c9698aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:32.725946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4451f-5cab-4697-ae2f-3df9c9698aa3 None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708/users/8fbd3678733b4165b9be465c46f4f48e/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:32.728101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4451f-5cab-4697-ae2f-3df9c9698aa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=52a0a003fc9e4f0ebf9e13f567cb8708, user_id=8fbd3678733b4165b9be465c46f4f48e, role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:32.729509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4451f-5cab-4697-ae2f-3df9c9698aa3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:32.842219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 568/1123] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:57:32 2022] PUT /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708/users/8fbd3678733b4165b9be465c46f4f48e/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:57:34.997883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968caa54-74b1-427d-b8ea-445d4a4a873f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:34.998439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968caa54-74b1-427d-b8ea-445d4a4a873f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:34.998439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968caa54-74b1-427d-b8ea-445d4a4a873f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:34.999816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 556/1124] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:35.005536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e58a2-0edf-4498-ad60-23810b1cb7f3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:35.005962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e58a2-0edf-4498-ad60-23810b1cb7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:35.006096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e58a2-0edf-4498-ad60-23810b1cb7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:35.022149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8e0e58a2-0edf-4498-ad60-23810b1cb7f3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:35.023326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8e0e58a2-0edf-4498-ad60-23810b1cb7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:35.086800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 569/1125] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:35.109571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968caa54-74b1-427d-b8ea-445d4a4a873f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:35.204756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968caa54-74b1-427d-b8ea-445d4a4a873f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:35.235776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:35.236620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:35.236896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:35.237169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] PATH_INFO: `/v3/roles/aa5c6f1dec664fb4b6274c2db76c4519` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:35.239241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] RBAC: Authorizing `identity:get_role(role_id=aa5c6f1dec664fb4b6274c2db76c4519)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:35.240618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:35.241980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 557/1126] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:35 2022] GET /identity/v3/roles/aa5c6f1dec664fb4b6274c2db76c4519 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:35.247872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e58a2-0edf-4498-ad60-23810b1cb7f3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:35.341812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e58a2-0edf-4498-ad60-23810b1cb7f3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:35.372984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:35.373789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:35.374069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:35.374338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] PATH_INFO: `/v3/users/8fbd3678733b4165b9be465c46f4f48e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:35.375486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] RBAC: Authorizing `identity:get_user(user_id=8fbd3678733b4165b9be465c46f4f48e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:35.376920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:35.379011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 570/1127] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:35 2022] GET /identity/v3/users/8fbd3678733b4165b9be465c46f4f48e => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:35.384533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:35.477139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef371a43-ab27-4372-a800-ac85303cd703 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:35.508785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:35.509414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:35.509697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:35.509973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:35.571897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:35.573325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:35.666239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 558/1128] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:35.671943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:35.766067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159181e-d951-4132-922f-b0d36f91a6c7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:35.797079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb3333-6734-4635-b499-d464fffb8779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:35.797915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb3333-6734-4635-b499-d464fffb8779 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:35.798193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb3333-6734-4635-b499-d464fffb8779 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:35.798465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb3333-6734-4635-b499-d464fffb8779 None admin] PATH_INFO: `/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:35.799616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb3333-6734-4635-b499-d464fffb8779 None admin] RBAC: Authorizing `identity:get_project(project_id=52a0a003fc9e4f0ebf9e13f567cb8708)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:35.801720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb3333-6734-4635-b499-d464fffb8779 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:35.803381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 571/1129] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:35 2022] GET /identity/v3/projects/52a0a003fc9e4f0ebf9e13f567cb8708 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:35.809292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:35.903165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a036e-e19a-451f-a205-30bf9d278bdb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:35.935200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46557a-3434-4d26-9b6e-a820d8c81d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:35.935965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46557a-3434-4d26-9b6e-a820d8c81d69 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:35.936247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46557a-3434-4d26-9b6e-a820d8c81d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:35.936518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46557a-3434-4d26-9b6e-a820d8c81d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:35.937337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46557a-3434-4d26-9b6e-a820d8c81d69 None admin] RBAC: Adding query filter params (role.id=aa5c6f1dec664fb4b6274c2db76c4519, scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708, user.id=8fbd3678733b4165b9be465c46f4f48e) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:35.937645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46557a-3434-4d26-9b6e-a820d8c81d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:35.939071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46557a-3434-4d26-9b6e-a820d8c81d69 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:35.945287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 559/1130] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:35 2022] GET /identity/v3/role_assignments?user.id=8fbd3678733b4165b9be465c46f4f48e&scope.project.id=52a0a003fc9e4f0ebf9e13f567cb8708&role.id=aa5c6f1dec664fb4b6274c2db76c4519 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:38.174191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d251cf0-f484-4008-bbdf-7f25b4df99b7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:38.174703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d251cf0-f484-4008-bbdf-7f25b4df99b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:38.174703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d251cf0-f484-4008-bbdf-7f25b4df99b7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:38.176190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 572/1131] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:38.181278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a6cd-de3b-43fc-ae0c-d4390f6e5f24 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:38.181691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a6cd-de3b-43fc-ae0c-d4390f6e5f24 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:38.181827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a6cd-de3b-43fc-ae0c-d4390f6e5f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:38.197989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2f88a6cd-de3b-43fc-ae0c-d4390f6e5f24 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:38.199153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2f88a6cd-de3b-43fc-ae0c-d4390f6e5f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:38.261061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 560/1132] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:38.284072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d251cf0-f484-4008-bbdf-7f25b4df99b7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:38.381310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d251cf0-f484-4008-bbdf-7f25b4df99b7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:38.414162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbfb6a-8497-4803-9a52-cac825d3695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:38.414944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbfb6a-8497-4803-9a52-cac825d3695d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:38.415227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbfb6a-8497-4803-9a52-cac825d3695d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:38.415500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbfb6a-8497-4803-9a52-cac825d3695d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:38.416685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbfb6a-8497-4803-9a52-cac825d3695d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:38.418117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbfb6a-8497-4803-9a52-cac825d3695d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:38.419752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 573/1133] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:38.426159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a6cd-de3b-43fc-ae0c-d4390f6e5f24 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:38.518550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a6cd-de3b-43fc-ae0c-d4390f6e5f24 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:38.550902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9978e-49c8-4225-8406-8c54eadad245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:38.551558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9978e-49c8-4225-8406-8c54eadad245 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:38.551864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9978e-49c8-4225-8406-8c54eadad245 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:38.552144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9978e-49c8-4225-8406-8c54eadad245 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:38.552843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9978e-49c8-4225-8406-8c54eadad245 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:38.554268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9978e-49c8-4225-8406-8c54eadad245 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:38.661552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 561/1134] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:57:38 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:40.859243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e1dd8-e740-40e5-839d-c977cc4edb24 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:40.859728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e1dd8-e740-40e5-839d-c977cc4edb24 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:40.859862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e1dd8-e740-40e5-839d-c977cc4edb24 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:40.861161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 574/1135] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:40.867151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea58f5-e33a-4b35-b80c-0bb11de466b3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:40.867578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea58f5-e33a-4b35-b80c-0bb11de466b3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:40.867730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea58f5-e33a-4b35-b80c-0bb11de466b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:40.884041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-17ea58f5-e33a-4b35-b80c-0bb11de466b3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:40.885198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-17ea58f5-e33a-4b35-b80c-0bb11de466b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:40.946229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 562/1136] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:40.970552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e1dd8-e740-40e5-839d-c977cc4edb24 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:41.068627 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e1dd8-e740-40e5-839d-c977cc4edb24 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:41.101431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1649e93-d100-4ab5-9ece-f7ed04873838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:41.102149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1649e93-d100-4ab5-9ece-f7ed04873838 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:41.102433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1649e93-d100-4ab5-9ece-f7ed04873838 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:41.102709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1649e93-d100-4ab5-9ece-f7ed04873838 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:41.103898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1649e93-d100-4ab5-9ece-f7ed04873838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:41.105322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1649e93-d100-4ab5-9ece-f7ed04873838 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:41.106929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 575/1137] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:57:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:41.112680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea58f5-e33a-4b35-b80c-0bb11de466b3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:41.205986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea58f5-e33a-4b35-b80c-0bb11de466b3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:41.237381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd0950-92dc-48e1-90f0-f4b7a6482597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:41.238587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0950-92dc-48e1-90f0-f4b7a6482597 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:41.238978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0950-92dc-48e1-90f0-f4b7a6482597 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:41.239258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0950-92dc-48e1-90f0-f4b7a6482597 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:41.239994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd0950-92dc-48e1-90f0-f4b7a6482597 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:41.241412 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd0950-92dc-48e1-90f0-f4b7a6482597 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:41.341934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 563/1138] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:57:41 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:43.610417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11748ca8-155e-4bc6-ba9e-8d62a111440b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:43.610977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11748ca8-155e-4bc6-ba9e-8d62a111440b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:43.610977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11748ca8-155e-4bc6-ba9e-8d62a111440b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:43.612348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 576/1139] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:43.617732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1cdfd-81ba-4f14-931b-a8982b935e32 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:43.618145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1cdfd-81ba-4f14-931b-a8982b935e32 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:43.618284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1cdfd-81ba-4f14-931b-a8982b935e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:43.635584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b4f1cdfd-81ba-4f14-931b-a8982b935e32 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:43.636776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b4f1cdfd-81ba-4f14-931b-a8982b935e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:43.697999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 564/1140] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:57:43.721468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11748ca8-155e-4bc6-ba9e-8d62a111440b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:43.817960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11748ca8-155e-4bc6-ba9e-8d62a111440b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:43.850646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:43.851429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:43.851744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:43.852017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:43.861767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:43.869974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:43.870941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:43.871878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 577/1141] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:57:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:43.877050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1cdfd-81ba-4f14-931b-a8982b935e32 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:43.968601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1cdfd-81ba-4f14-931b-a8982b935e32 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:44.000352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:44.000994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:44.001272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:44.001542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:44.002273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:44.002573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:44.004062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:44.011687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 565/1142] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:57:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 14:57:44.017329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:44.111470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565470b-33c8-4a00-a663-520d055d576d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:44.142798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:44.143617 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:44.143923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:44.144195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] PATH_INFO: `/v3/users/ef521d4e323b4300810980deaa6e93f6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:44.157102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] RBAC: Authorizing `identity:get_user(user_id=ef521d4e323b4300810980deaa6e93f6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:44.158520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:44.160713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 578/1143] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:44 2022] GET /identity/v3/users/ef521d4e323b4300810980deaa6e93f6 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:44.166286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:44.259467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74251b-a21d-4933-9224-a140487eaae2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:44.291386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:44.292053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:44.292336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:44.292608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:44.356489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:44.357923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:44.449444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 566/1144] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:44.455181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:44.548406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91409cc5-35f6-4558-a10d-bcbdf7b4ee16 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:44.581040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23285b-2c94-4780-b8b9-7c6175c0799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:44.581880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23285b-2c94-4780-b8b9-7c6175c0799b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:44.582903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23285b-2c94-4780-b8b9-7c6175c0799b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:44.583239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23285b-2c94-4780-b8b9-7c6175c0799b None admin] PATH_INFO: `/v3/projects/f47ea6ec28fe42c38283eede435c5598` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:44.584434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23285b-2c94-4780-b8b9-7c6175c0799b None admin] RBAC: Authorizing `identity:get_project(project_id=f47ea6ec28fe42c38283eede435c5598)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:44.585851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23285b-2c94-4780-b8b9-7c6175c0799b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:44.587519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 579/1145] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:44 2022] GET /identity/v3/projects/f47ea6ec28fe42c38283eede435c5598 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:44.593442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:44.685852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc888c6b-2b19-4ef0-851e-d369cafda006 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:44.717533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9600e-e402-4862-bcb9-39be060b9661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:44.718107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9600e-e402-4862-bcb9-39be060b9661 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:44.718387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9600e-e402-4862-bcb9-39be060b9661 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:44.718658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9600e-e402-4862-bcb9-39be060b9661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:44.719478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9600e-e402-4862-bcb9-39be060b9661 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=f47ea6ec28fe42c38283eede435c5598, user.id=ef521d4e323b4300810980deaa6e93f6) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:44.719812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9600e-e402-4862-bcb9-39be060b9661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:44.721223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9600e-e402-4862-bcb9-39be060b9661 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:44.726726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 567/1146] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:44 2022] GET /identity/v3/role_assignments?user.id=ef521d4e323b4300810980deaa6e93f6&scope.project.id=f47ea6ec28fe42c38283eede435c5598&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:46.909233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f639b1-ef5b-4df8-ae4b-b3011e9d7e31 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:46.909715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f639b1-ef5b-4df8-ae4b-b3011e9d7e31 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:46.909837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f639b1-ef5b-4df8-ae4b-b3011e9d7e31 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:46.911795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 580/1147] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:46.916982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b27f-dbf9-4d21-8ce4-6d23f9bba733 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:46.917360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b27f-dbf9-4d21-8ce4-6d23f9bba733 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:46.917533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b27f-dbf9-4d21-8ce4-6d23f9bba733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:46.934223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b979b27f-dbf9-4d21-8ce4-6d23f9bba733 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:46.935395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b979b27f-dbf9-4d21-8ce4-6d23f9bba733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:46.998728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 568/1148] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:47.021339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f639b1-ef5b-4df8-ae4b-b3011e9d7e31 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:47.117380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f639b1-ef5b-4df8-ae4b-b3011e9d7e31 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:47.148972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:47.149774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:47.150055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:47.150332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:47.159815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:47.168213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:47.169280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:47.170185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 581/1149] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:57:47.175237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979b27f-dbf9-4d21-8ce4-6d23f9bba733 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:47.266707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979b27f-dbf9-4d21-8ce4-6d23f9bba733 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:47.298817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:47.299468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:47.299778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:47.300055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:47.300797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:47.301247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:47.302622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:47.310145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 569/1150] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:47.316090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:47.411442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c96c-ca4c-469f-b7f9-5ae97210f74d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:47.442900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:47.443903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:47.444191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:47.444468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] PATH_INFO: `/v3/users/ef521d4e323b4300810980deaa6e93f6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:47.445632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ef521d4e323b4300810980deaa6e93f6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:47.447078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:47.448703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 582/1151] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:47 2022] GET /identity/v3/users/ef521d4e323b4300810980deaa6e93f6 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:47.454213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:47.547358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7c214-9df8-4516-9e54-6a5cf598d15a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:47.578171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa2fa9-93cb-470e-b967-1182491a30cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:47.579010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2fa9-93cb-470e-b967-1182491a30cc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:47.579290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2fa9-93cb-470e-b967-1182491a30cc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:47.579563 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2fa9-93cb-470e-b967-1182491a30cc None admin] PATH_INFO: `/v3/projects/f47ea6ec28fe42c38283eede435c5598` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:47.580742 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa2fa9-93cb-470e-b967-1182491a30cc None admin] RBAC: Authorizing `identity:get_project(project_id=f47ea6ec28fe42c38283eede435c5598)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:47.582155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa2fa9-93cb-470e-b967-1182491a30cc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:47.583949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 570/1152] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:47 2022] GET /identity/v3/projects/f47ea6ec28fe42c38283eede435c5598 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:47.589722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:47.683921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca2d52-77c0-43a5-bd96-96bf41db05d4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:47.715386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d5809-8fef-400b-9959-ebea85ec2627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:47.716288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d5809-8fef-400b-9959-ebea85ec2627 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:47.716566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d5809-8fef-400b-9959-ebea85ec2627 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:47.716834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d5809-8fef-400b-9959-ebea85ec2627 None admin] PATH_INFO: `/v3/projects/f47ea6ec28fe42c38283eede435c5598/users/ef521d4e323b4300810980deaa6e93f6/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:47.718731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d5809-8fef-400b-9959-ebea85ec2627 None admin] RBAC: Authorizing `identity:create_grant(project_id=f47ea6ec28fe42c38283eede435c5598, user_id=ef521d4e323b4300810980deaa6e93f6, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:47.720177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d5809-8fef-400b-9959-ebea85ec2627 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:47.839645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 583/1153] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:57:47 2022] PUT /identity/v3/projects/f47ea6ec28fe42c38283eede435c5598/users/ef521d4e323b4300810980deaa6e93f6/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:57:50.057693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82502c2e-e39c-4b11-897f-842579333110 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:50.058237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82502c2e-e39c-4b11-897f-842579333110 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:50.058237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82502c2e-e39c-4b11-897f-842579333110 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:50.059578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 571/1154] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:50.064723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa372a-90aa-4ac1-9caa-bc165cd1325a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:50.065116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa372a-90aa-4ac1-9caa-bc165cd1325a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:50.065271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa372a-90aa-4ac1-9caa-bc165cd1325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:50.081608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d6aa372a-90aa-4ac1-9caa-bc165cd1325a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:50.083783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d6aa372a-90aa-4ac1-9caa-bc165cd1325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:50.147150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 584/1155] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:50.170706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82502c2e-e39c-4b11-897f-842579333110 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:50.265564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82502c2e-e39c-4b11-897f-842579333110 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:50.297134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:50.297900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:50.298176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:50.298441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:50.307939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:57:50.315414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:50.316536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:57:50.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 572/1156] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:57:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:50.323158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa372a-90aa-4ac1-9caa-bc165cd1325a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:50.416176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa372a-90aa-4ac1-9caa-bc165cd1325a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:50.448011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:50.448665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:50.448944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:50.449214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:50.449949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:50.450245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:50.451597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:50.459806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 585/1157] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:57:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:50.465422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:50.558976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92017ed-e0b9-476c-bfd7-feaa5ac813bd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:50.591599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:50.592474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:50.592752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:50.593021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] PATH_INFO: `/v3/users/ef521d4e323b4300810980deaa6e93f6` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:50.594178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] RBAC: Authorizing `identity:get_user(user_id=ef521d4e323b4300810980deaa6e93f6)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:50.595607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:50.597236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 573/1158] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:57:50 2022] GET /identity/v3/users/ef521d4e323b4300810980deaa6e93f6 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:50.603002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:50.696736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551f68-abd8-4c99-8b93-da62acea983c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:50.728174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:50.728805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:50.729088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:50.729359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:50.791926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:50.793362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:50.886019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 586/1159] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:50.891903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:50.983561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be72ba5-f6ed-4d09-b997-3db213b3028a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:51.015007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3168-77f5-4798-a571-159c31a1a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:51.015863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3168-77f5-4798-a571-159c31a1a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:51.016143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3168-77f5-4798-a571-159c31a1a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:51.016410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3168-77f5-4798-a571-159c31a1a6c0 None admin] PATH_INFO: `/v3/projects/f47ea6ec28fe42c38283eede435c5598` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:51.017549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3168-77f5-4798-a571-159c31a1a6c0 None admin] RBAC: Authorizing `identity:get_project(project_id=f47ea6ec28fe42c38283eede435c5598)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:51.018965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3168-77f5-4798-a571-159c31a1a6c0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:51.020800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 574/1160] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:57:50 2022] GET /identity/v3/projects/f47ea6ec28fe42c38283eede435c5598 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:51.026702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:51.129503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaee7d6-910c-4895-8c62-72cd0c696b55 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:51.161507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d44a49-ac38-4a49-b571-67f89cd38dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:51.162144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d44a49-ac38-4a49-b571-67f89cd38dbd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:51.162430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d44a49-ac38-4a49-b571-67f89cd38dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:51.162707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d44a49-ac38-4a49-b571-67f89cd38dbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:51.163527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d44a49-ac38-4a49-b571-67f89cd38dbd None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=f47ea6ec28fe42c38283eede435c5598, user.id=ef521d4e323b4300810980deaa6e93f6) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:51.163866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d44a49-ac38-4a49-b571-67f89cd38dbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:51.165288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d44a49-ac38-4a49-b571-67f89cd38dbd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:51.170845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 587/1161] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:57:51 2022] GET /identity/v3/role_assignments?user.id=ef521d4e323b4300810980deaa6e93f6&scope.project.id=f47ea6ec28fe42c38283eede435c5598&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:53.392750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210c42-242f-42d1-a319-244b9a4fcb98 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:53.393340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210c42-242f-42d1-a319-244b9a4fcb98 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:53.393340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210c42-242f-42d1-a319-244b9a4fcb98 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:53.394660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 575/1162] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:53.400421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3d9dc-4c2c-4db2-afa7-83c678362549 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:53.400807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3d9dc-4c2c-4db2-afa7-83c678362549 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:53.400978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3d9dc-4c2c-4db2-afa7-83c678362549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:53.418017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f7f3d9dc-4c2c-4db2-afa7-83c678362549 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:53.419181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7f3d9dc-4c2c-4db2-afa7-83c678362549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:53.483049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 588/1163] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:53.505092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210c42-242f-42d1-a319-244b9a4fcb98 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:53.600792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210c42-242f-42d1-a319-244b9a4fcb98 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:53.631924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:53.632551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:53.632827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:53.633094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:53.694770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:53.696157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:53.785623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 576/1164] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:57:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:57:53.792145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3d9dc-4c2c-4db2-afa7-83c678362549 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:53.884650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3d9dc-4c2c-4db2-afa7-83c678362549 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:53.915882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc0ab5-723b-416d-a384-5bf4cedac3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:53.916670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc0ab5-723b-416d-a384-5bf4cedac3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:53.916946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc0ab5-723b-416d-a384-5bf4cedac3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:53.917216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc0ab5-723b-416d-a384-5bf4cedac3f1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:53.923518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc0ab5-723b-416d-a384-5bf4cedac3f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:53.924967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc0ab5-723b-416d-a384-5bf4cedac3f1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-17dc0ab5-723b-416d-a384-5bf4cedac3f1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:57:53.939014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:57:53.940330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:57:53.941467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 589/1165] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:57:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:57:53.946944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:54.037950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2503ed-f899-42ad-92fe-4e875e46e556 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:54.070714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e0fc8-817e-49ff-8e6b-c75120adfdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:54.071363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0fc8-817e-49ff-8e6b-c75120adfdea None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:54.071678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0fc8-817e-49ff-8e6b-c75120adfdea None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:54.071960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0fc8-817e-49ff-8e6b-c75120adfdea None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:54.072675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e0fc8-817e-49ff-8e6b-c75120adfdea None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:57:54.072981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e0fc8-817e-49ff-8e6b-c75120adfdea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:54.074339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e0fc8-817e-49ff-8e6b-c75120adfdea None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:54.081193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 577/1166] 10.222.0.71 () {56 vars in 1115 bytes} [Mon May 16 14:57:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:56.273202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90470ddb-0794-4991-9d96-2cf18650d4fc None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:56.273756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90470ddb-0794-4991-9d96-2cf18650d4fc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:56.273756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90470ddb-0794-4991-9d96-2cf18650d4fc None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:56.275118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 590/1167] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:56.280408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3bc09-8aaf-49b8-90c2-63d2a92ee2f1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:56.280841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3bc09-8aaf-49b8-90c2-63d2a92ee2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:56.280968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3bc09-8aaf-49b8-90c2-63d2a92ee2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:56.297660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-26c3bc09-8aaf-49b8-90c2-63d2a92ee2f1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:56.298837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-26c3bc09-8aaf-49b8-90c2-63d2a92ee2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:56.362336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 578/1168] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:56.385288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90470ddb-0794-4991-9d96-2cf18650d4fc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:56.482487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90470ddb-0794-4991-9d96-2cf18650d4fc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:56.514057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e7127-09bf-4cd0-be5c-0ead796c8fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:56.514704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7127-09bf-4cd0-be5c-0ead796c8fe5 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:56.514988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7127-09bf-4cd0-be5c-0ead796c8fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:56.515266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7127-09bf-4cd0-be5c-0ead796c8fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:56.516029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e7127-09bf-4cd0-be5c-0ead796c8fe5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:56.517394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e7127-09bf-4cd0-be5c-0ead796c8fe5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:56.628003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 591/1169] 10.222.0.71 () {60 vars in 1138 bytes} [Mon May 16 14:57:56 2022] POST /identity/v3/domains => generated 267 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:57:58.837602 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb9acc-47af-4dfd-8795-391b0f2155a5 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:58.838167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb9acc-47af-4dfd-8795-391b0f2155a5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:58.838167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb9acc-47af-4dfd-8795-391b0f2155a5 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:58.839504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 579/1170] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:57:58.844999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c2be2-8dfc-47d0-90c9-06491159b1e9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:58.845367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c2be2-8dfc-47d0-90c9-06491159b1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:58.845567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c2be2-8dfc-47d0-90c9-06491159b1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:58.862764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-524c2be2-8dfc-47d0-90c9-06491159b1e9 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:57:58.863970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-524c2be2-8dfc-47d0-90c9-06491159b1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:57:58.927430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 592/1171] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:57:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:57:58.951100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb9acc-47af-4dfd-8795-391b0f2155a5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:59.045702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb9acc-47af-4dfd-8795-391b0f2155a5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:59.086694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b250b3-28ae-4939-b948-fb17290c0530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:59.087539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b250b3-28ae-4939-b948-fb17290c0530 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:59.087897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b250b3-28ae-4939-b948-fb17290c0530 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:59.088119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b250b3-28ae-4939-b948-fb17290c0530 None admin] PATH_INFO: `/v3/domains/31625c6ca2fc43cab7658f32cf1a2d2e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:59.098184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b250b3-28ae-4939-b948-fb17290c0530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=31625c6ca2fc43cab7658f32cf1a2d2e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:59.100004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b250b3-28ae-4939-b948-fb17290c0530 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:59.101641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 580/1172] 10.222.0.71 () {56 vars in 1183 bytes} [Mon May 16 14:57:58 2022] GET /identity/v3/domains/31625c6ca2fc43cab7658f32cf1a2d2e => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:57:59.107386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c2be2-8dfc-47d0-90c9-06491159b1e9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:57:59.201103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c2be2-8dfc-47d0-90c9-06491159b1e9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:57:59.232707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa9f97-0065-4999-a9a4-5cca265a43b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:57:59.233354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9f97-0065-4999-a9a4-5cca265a43b5 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:57:59.233637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9f97-0065-4999-a9a4-5cca265a43b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:57:59.233915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9f97-0065-4999-a9a4-5cca265a43b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:57:59.234611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa9f97-0065-4999-a9a4-5cca265a43b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:57:59.236072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa9f97-0065-4999-a9a4-5cca265a43b5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:57:59.327550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 593/1173] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:57:59 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:58:01.575074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6258c-f0c2-4c08-b6fc-91fff74f5675 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:01.575670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6258c-f0c2-4c08-b6fc-91fff74f5675 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:01.575670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6258c-f0c2-4c08-b6fc-91fff74f5675 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:01.577013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 581/1174] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:58:01.582176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cb66f-7e33-4432-9255-e5ee1cfaa1f8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:01.582582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cb66f-7e33-4432-9255-e5ee1cfaa1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:01.582735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cb66f-7e33-4432-9255-e5ee1cfaa1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:01.600053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e18cb66f-7e33-4432-9255-e5ee1cfaa1f8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:58:01.601255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e18cb66f-7e33-4432-9255-e5ee1cfaa1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:58:01.664872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 594/1175] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:58:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:58:01.689611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6258c-f0c2-4c08-b6fc-91fff74f5675 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:01.784547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6258c-f0c2-4c08-b6fc-91fff74f5675 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:01.816619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cd516-3170-4b1a-a714-6bd62fa5c8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:01.817459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cd516-3170-4b1a-a714-6bd62fa5c8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:01.817783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cd516-3170-4b1a-a714-6bd62fa5c8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:01.818123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cd516-3170-4b1a-a714-6bd62fa5c8b2 None admin] PATH_INFO: `/v3/domains/31625c6ca2fc43cab7658f32cf1a2d2e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:01.819278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cd516-3170-4b1a-a714-6bd62fa5c8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=31625c6ca2fc43cab7658f32cf1a2d2e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:01.820720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cd516-3170-4b1a-a714-6bd62fa5c8b2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:01.822309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 582/1176] 10.222.0.71 () {56 vars in 1183 bytes} [Mon May 16 14:58:01 2022] GET /identity/v3/domains/31625c6ca2fc43cab7658f32cf1a2d2e => generated 267 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:01.827961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cb66f-7e33-4432-9255-e5ee1cfaa1f8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:01.921628 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cb66f-7e33-4432-9255-e5ee1cfaa1f8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:01.953685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078c633-7466-4b3c-8a3b-ff34eb1eb775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:01.954341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c633-7466-4b3c-8a3b-ff34eb1eb775 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:01.954623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c633-7466-4b3c-8a3b-ff34eb1eb775 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:01.954898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078c633-7466-4b3c-8a3b-ff34eb1eb775 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:01.955595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c633-7466-4b3c-8a3b-ff34eb1eb775 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:01.957110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078c633-7466-4b3c-8a3b-ff34eb1eb775 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:02.061609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 595/1177] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:58:01 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:58:04.329242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22989871-f0b7-4a3f-82dd-5e9c113bf63a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:04.338899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22989871-f0b7-4a3f-82dd-5e9c113bf63a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:04.338899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22989871-f0b7-4a3f-82dd-5e9c113bf63a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:04.338899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 583/1178] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:58:04.339160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bc070-5b78-4d69-bf57-3e0ca734d992 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:04.339160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bc070-5b78-4d69-bf57-3e0ca734d992 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:04.339160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bc070-5b78-4d69-bf57-3e0ca734d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:04.354916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e05bc070-5b78-4d69-bf57-3e0ca734d992 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:58:04.356097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e05bc070-5b78-4d69-bf57-3e0ca734d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:58:04.423137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 596/1179] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:58:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:58:04.446252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22989871-f0b7-4a3f-82dd-5e9c113bf63a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:04.542179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22989871-f0b7-4a3f-82dd-5e9c113bf63a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:04.574076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:04.574846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:04.575129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:04.575405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:04.585789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:58:04.593325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:58:04.594444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:58:04.595338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 584/1180] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:58:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:58:04.601193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bc070-5b78-4d69-bf57-3e0ca734d992 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:04.694911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bc070-5b78-4d69-bf57-3e0ca734d992 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:04.727107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:04.727792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:04.728078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:04.728353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:04.729092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:58:04.729396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:04.730774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:04.738466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 597/1181] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:58:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:04.745127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:04.838700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dbaa4-5655-4cf9-9653-a1b9f45a01cf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:04.870736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:04.871602 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:04.871913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:04.872190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] PATH_INFO: `/v3/users/d076e3ace53642b0a5182eb54f4efa93` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:04.884863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=d076e3ace53642b0a5182eb54f4efa93)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:04.886295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:04.887974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 585/1182] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:58:04 2022] GET /identity/v3/users/d076e3ace53642b0a5182eb54f4efa93 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:04.893601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:04.989373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61467b-45a7-44de-99c3-449b97598e9a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:05.020157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:05.020792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:05.021674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:05.021987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:05.084193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:05.085693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:05.177828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 598/1183] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:58:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:58:05.183551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:05.275065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d04a85-2908-4c80-8481-890d6a380ea9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:05.306725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd23367-4619-4e74-941a-148d840e1ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:05.307559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23367-4619-4e74-941a-148d840e1ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:05.307861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23367-4619-4e74-941a-148d840e1ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:05.308129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23367-4619-4e74-941a-148d840e1ae3 None admin] PATH_INFO: `/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:05.309272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd23367-4619-4e74-941a-148d840e1ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=6e798925a38a4d48b7e9db0b0e769bb5)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:05.310685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd23367-4619-4e74-941a-148d840e1ae3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:05.312391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 586/1184] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:58:05 2022] GET /identity/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5 => generated 361 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:05.319017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:05.412598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fb420-59c3-435e-8cbc-e1d1fe5bdbc5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:05.444785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d7aef-3bb7-40aa-aa29-3acea968c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:05.445364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d7aef-3bb7-40aa-aa29-3acea968c863 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:05.445642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d7aef-3bb7-40aa-aa29-3acea968c863 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:05.445914 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d7aef-3bb7-40aa-aa29-3acea968c863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:05.446731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d7aef-3bb7-40aa-aa29-3acea968c863 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=6e798925a38a4d48b7e9db0b0e769bb5, user.id=d076e3ace53642b0a5182eb54f4efa93) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:58:05.447041 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d7aef-3bb7-40aa-aa29-3acea968c863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:05.448478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d7aef-3bb7-40aa-aa29-3acea968c863 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:05.453615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 599/1185] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:58:05 2022] GET /identity/v3/role_assignments?user.id=d076e3ace53642b0a5182eb54f4efa93&scope.project.id=6e798925a38a4d48b7e9db0b0e769bb5&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:07.688966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697cb0c-0ef2-4a95-aca8-752870091d91 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:07.737217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697cb0c-0ef2-4a95-aca8-752870091d91 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:07.737217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697cb0c-0ef2-4a95-aca8-752870091d91 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:07.737217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 587/1186] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:58:07.737535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8fc87-7b28-414f-8828-ffa5b22955ff None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:07.737535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8fc87-7b28-414f-8828-ffa5b22955ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:07.737535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8fc87-7b28-414f-8828-ffa5b22955ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:07.737535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-54e8fc87-7b28-414f-8828-ffa5b22955ff None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:58:07.737535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-54e8fc87-7b28-414f-8828-ffa5b22955ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:58:07.781829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 600/1187] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:58:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:58:07.805696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697cb0c-0ef2-4a95-aca8-752870091d91 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:07.902255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697cb0c-0ef2-4a95-aca8-752870091d91 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:07.933674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:07.934445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:07.934721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:07.934989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:07.945201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:58:07.952598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:58:07.953782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:58:07.954963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 588/1188] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:58:07.959988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8fc87-7b28-414f-8828-ffa5b22955ff None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:08.051032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8fc87-7b28-414f-8828-ffa5b22955ff None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:08.083043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:08.083716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:08.084199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:08.084481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:08.085215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:58:08.085517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:08.086872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:08.094480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 601/1189] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:58:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:08.100406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:08.193049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091003fb-5b88-4474-bfa6-672d103dd01f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:08.224816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:08.225658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:08.225928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:08.226195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] PATH_INFO: `/v3/users/d076e3ace53642b0a5182eb54f4efa93` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:08.227340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] RBAC: Authorizing `identity:get_user(user_id=d076e3ace53642b0a5182eb54f4efa93)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:08.228812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:08.230392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 589/1190] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:58:08 2022] GET /identity/v3/users/d076e3ace53642b0a5182eb54f4efa93 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:08.235980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:08.332863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306d54e-a1cd-45d5-b345-8cd3035f4adb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:08.365694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf64b57-c7f9-4876-9366-31868d671140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:08.367158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf64b57-c7f9-4876-9366-31868d671140 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:08.367436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf64b57-c7f9-4876-9366-31868d671140 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:08.367736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf64b57-c7f9-4876-9366-31868d671140 None admin] PATH_INFO: `/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:08.368901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf64b57-c7f9-4876-9366-31868d671140 None admin] RBAC: Authorizing `identity:get_project(project_id=6e798925a38a4d48b7e9db0b0e769bb5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:08.370325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf64b57-c7f9-4876-9366-31868d671140 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:08.372013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 602/1191] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:58:08 2022] GET /identity/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:08.377813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:08.470262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8b05d-1de5-41cd-8ac3-bd388fb7eba3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:08.501883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ffc7a-e653-484d-9649-7309f82779ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:08.502687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ffc7a-e653-484d-9649-7309f82779ed None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:08.502966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ffc7a-e653-484d-9649-7309f82779ed None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:08.503239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ffc7a-e653-484d-9649-7309f82779ed None admin] PATH_INFO: `/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5/users/d076e3ace53642b0a5182eb54f4efa93/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:08.505178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ffc7a-e653-484d-9649-7309f82779ed None admin] RBAC: Authorizing `identity:create_grant(project_id=6e798925a38a4d48b7e9db0b0e769bb5, user_id=d076e3ace53642b0a5182eb54f4efa93, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:08.506639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ffc7a-e653-484d-9649-7309f82779ed None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:08.583516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 590/1192] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:58:08 2022] PUT /identity/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5/users/d076e3ace53642b0a5182eb54f4efa93/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:58:10.752625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb221d-dc6f-4e9f-9f49-6b14b40a5b76 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:10.753182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb221d-dc6f-4e9f-9f49-6b14b40a5b76 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:10.753182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb221d-dc6f-4e9f-9f49-6b14b40a5b76 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:10.754514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 603/1193] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:58:10.759998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8872-97c9-4cc3-8e60-ce6c8ad94809 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:10.760420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8872-97c9-4cc3-8e60-ce6c8ad94809 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:10.760555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8872-97c9-4cc3-8e60-ce6c8ad94809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:10.776810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-70bb8872-97c9-4cc3-8e60-ce6c8ad94809 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:58:10.777982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-70bb8872-97c9-4cc3-8e60-ce6c8ad94809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:58:10.841450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 591/1194] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:58:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:58:10.865197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb221d-dc6f-4e9f-9f49-6b14b40a5b76 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:10.962485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb221d-dc6f-4e9f-9f49-6b14b40a5b76 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:10.994098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:10.994873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:10.995154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:10.995428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:11.005398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:58:11.012886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:58:11.013995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:58:11.015042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 604/1195] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:58:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:58:11.020175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb8872-97c9-4cc3-8e60-ce6c8ad94809 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:11.112866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb8872-97c9-4cc3-8e60-ce6c8ad94809 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:11.145388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:11.146045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:11.146329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:11.147168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:11.147937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:58:11.148245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:11.149603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:11.157353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 592/1196] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:11.163940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:11.257170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ea02b-4435-4527-8222-ec851c119f3f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:11.289938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:11.290796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:11.291083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:11.291357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] PATH_INFO: `/v3/users/d076e3ace53642b0a5182eb54f4efa93` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:11.292543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] RBAC: Authorizing `identity:get_user(user_id=d076e3ace53642b0a5182eb54f4efa93)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:11.293962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:11.295553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 605/1197] 10.222.0.71 () {56 vars in 1177 bytes} [Mon May 16 14:58:11 2022] GET /identity/v3/users/d076e3ace53642b0a5182eb54f4efa93 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:11.301055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:11.398527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0632073-d17c-4e7b-87f8-b0b774844646 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:11.429915 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:11.430551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:11.430835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:11.431113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:11.494658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:11.496118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:11.589508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 593/1198] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:58:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:58:11.595472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:11.690150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb52bee-d27d-41b9-bf79-703d64707b03 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:11.723108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffda62b-a2ad-4e8a-9e27-961a872e20f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:11.723972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffda62b-a2ad-4e8a-9e27-961a872e20f7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:11.724253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffda62b-a2ad-4e8a-9e27-961a872e20f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:11.724528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffda62b-a2ad-4e8a-9e27-961a872e20f7 None admin] PATH_INFO: `/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:11.726450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffda62b-a2ad-4e8a-9e27-961a872e20f7 None admin] RBAC: Authorizing `identity:get_project(project_id=6e798925a38a4d48b7e9db0b0e769bb5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:11.727892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffda62b-a2ad-4e8a-9e27-961a872e20f7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:11.729629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 606/1199] 10.222.0.71 () {56 vars in 1186 bytes} [Mon May 16 14:58:11 2022] GET /identity/v3/projects/6e798925a38a4d48b7e9db0b0e769bb5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:58:11.735502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:58:11.829881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756db6eb-fa50-4e5a-9d85-a62eeba98fda None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:58:11.863235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00632a46-61ba-41f5-aefd-a499473214fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:58:11.863913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632a46-61ba-41f5-aefd-a499473214fe None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:58:11.864192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632a46-61ba-41f5-aefd-a499473214fe None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:58:11.864461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632a46-61ba-41f5-aefd-a499473214fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:58:11.865275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00632a46-61ba-41f5-aefd-a499473214fe None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=6e798925a38a4d48b7e9db0b0e769bb5, user.id=d076e3ace53642b0a5182eb54f4efa93) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:58:11.866388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00632a46-61ba-41f5-aefd-a499473214fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:58:11.867582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00632a46-61ba-41f5-aefd-a499473214fe None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:58:11.873207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 594/1200] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:58:11 2022] GET /identity/v3/role_assignments?user.id=d076e3ace53642b0a5182eb54f4efa93&scope.project.id=6e798925a38a4d48b7e9db0b0e769bb5&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:28.098141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92adfd-9f4f-4d9f-8268-c3718dd9fe9c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:28.098689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92adfd-9f4f-4d9f-8268-c3718dd9fe9c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:28.098689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92adfd-9f4f-4d9f-8268-c3718dd9fe9c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:28.100607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 607/1201] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:28.106000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d80b2-e90f-4f4a-8a0b-5fb611cbd967 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:28.106422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d80b2-e90f-4f4a-8a0b-5fb611cbd967 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:28.106555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d80b2-e90f-4f4a-8a0b-5fb611cbd967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:28.123320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-437d80b2-e90f-4f4a-8a0b-5fb611cbd967 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:28.124623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-437d80b2-e90f-4f4a-8a0b-5fb611cbd967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:28.187922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 595/1202] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:28.213339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92adfd-9f4f-4d9f-8268-c3718dd9fe9c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:28.311813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92adfd-9f4f-4d9f-8268-c3718dd9fe9c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:28.345047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:28.345822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:28.346099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:28.346370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:28.356057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:28.357419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:28.374761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:28.375904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:28.377074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:28.377074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:28.377074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:28.377074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:28.377074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 608/1203] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:28.382664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d80b2-e90f-4f4a-8a0b-5fb611cbd967 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:28.476646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d80b2-e90f-4f4a-8a0b-5fb611cbd967 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:28.510337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaf3e1-8714-4097-9a33-5857b389b468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:28.511323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaf3e1-8714-4097-9a33-5857b389b468 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:28.511323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaf3e1-8714-4097-9a33-5857b389b468 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:28.511511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaf3e1-8714-4097-9a33-5857b389b468 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:28.512276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaf3e1-8714-4097-9a33-5857b389b468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:28.512587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaf3e1-8714-4097-9a33-5857b389b468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:28.513943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaf3e1-8714-4097-9a33-5857b389b468 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:28.525236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 596/1204] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:28.530861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:28.627666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346ab0d-3e0e-4777-9538-c92291422f59 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:28.660712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9d66a-5d4c-48f3-8b4a-70e1487480b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:28.661493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9d66a-5d4c-48f3-8b4a-70e1487480b1 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:28.661923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9d66a-5d4c-48f3-8b4a-70e1487480b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:28.662201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9d66a-5d4c-48f3-8b4a-70e1487480b1 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:28.662903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9d66a-5d4c-48f3-8b4a-70e1487480b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:28.664417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9d66a-5d4c-48f3-8b4a-70e1487480b1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:28.741415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 609/1205] 10.222.0.71 () {60 vars in 1132 bytes} [Mon May 16 14:59:28 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 14:59:30.958760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160290b-39a2-432a-bf67-c0384d6dcca3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:30.959348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160290b-39a2-432a-bf67-c0384d6dcca3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:30.959348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160290b-39a2-432a-bf67-c0384d6dcca3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:30.960688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 597/1206] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:30.965929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f2fb6-ba83-44eb-bcc7-c85f154f5cd4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:30.966363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f2fb6-ba83-44eb-bcc7-c85f154f5cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:30.966467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f2fb6-ba83-44eb-bcc7-c85f154f5cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:30.983157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ad0f2fb6-ba83-44eb-bcc7-c85f154f5cd4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:30.984361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ad0f2fb6-ba83-44eb-bcc7-c85f154f5cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:31.048656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 610/1207] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 14:59:31.072492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160290b-39a2-432a-bf67-c0384d6dcca3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:31.169624 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160290b-39a2-432a-bf67-c0384d6dcca3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:31.201415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:31.202185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:31.202462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:31.202728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:31.212796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:59:31.220546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:31.227118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:31.228069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 598/1208] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:59:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:31.228952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f2fb6-ba83-44eb-bcc7-c85f154f5cd4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:31.323352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f2fb6-ba83-44eb-bcc7-c85f154f5cd4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:31.355351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:31.356044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:31.356352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:31.356630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:31.357369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:31.357674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:31.359038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:31.368241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 611/1209] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:59:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:31.374076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:31.469574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa40223-3810-4571-9032-6ec12ea76821 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:31.501547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:31.502324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:31.502605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:31.502876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:31.514540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:31.516003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:31.532577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:31.533652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:31.534769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:31.534769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:31.534769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:31.534769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:31.534769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 599/1210] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:31.539618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:31.637426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1285f0-05bb-4212-9d35-59aa0d3a8521 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:31.669008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:31.669665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:31.669943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:31.670215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:31.670925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:31.671223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:31.672647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:31.684234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 612/1211] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:31.690144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:31.786078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a5260-2b22-49e3-958d-91342c9dfebc None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:31.818015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:31.818861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:31.819141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:31.819414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:31.827046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:31.828505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:59:31.843255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:31.844565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:31.846478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 600/1212] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:31.850828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:31.946927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bae70-0deb-49ef-b50e-838034f9e463 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:31.980328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:31.980988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:31.981268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:31.981541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:31.982408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:31.982713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:31.984169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:32.001618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 613/1213] 10.222.0.71 () {56 vars in 1143 bytes} [Mon May 16 14:59:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:32.007417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:32.109477 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f381b2-aaf6-41d8-b13d-628c00c73a0a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:32.141378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:32.142017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:32.142296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:32.142569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:32.208049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:32.209582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:32.303476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 601/1214] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:59:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:59:32.309507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:32.404814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c456256-d314-4de8-b75a-3365db1390e1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:32.438468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78977-95ad-45ad-8697-25595202651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:32.439245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78977-95ad-45ad-8697-25595202651f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:32.439527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78977-95ad-45ad-8697-25595202651f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:32.441492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78977-95ad-45ad-8697-25595202651f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:32.450898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f78977-95ad-45ad-8697-25595202651f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:32.452367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f78977-95ad-45ad-8697-25595202651f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-40f78977-95ad-45ad-8697-25595202651f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:32.469806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:32.471117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:32.472366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:32.472366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:32.472366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:32.472366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:32.472366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 614/1215] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:32.477466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:32.571456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc9c82-b5e7-4400-a12f-51083146a39f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:32.603704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:32.604357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:32.604636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:32.604907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:32.605615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:32.605916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:32.607275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:32.618038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 602/1216] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:32.623663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78977-95ad-45ad-8697-25595202651f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:32.720912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78977-95ad-45ad-8697-25595202651f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:32.754335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:32.755299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:32.755577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:32.755876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:32.762489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:32.763936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:59:32.777917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:32.778817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:32.779833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 615/1217] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:32.785032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:32.883754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093100-5504-465c-98b5-5392e12a0e83 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:32.915179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dade4-c4c3-45a3-a818-1bd6972d852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:32.915859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dade4-c4c3-45a3-a818-1bd6972d852b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:32.916139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dade4-c4c3-45a3-a818-1bd6972d852b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:32.916408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dade4-c4c3-45a3-a818-1bd6972d852b None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:32.917184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dade4-c4c3-45a3-a818-1bd6972d852b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:32.917490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dade4-c4c3-45a3-a818-1bd6972d852b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:32.918895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dade4-c4c3-45a3-a818-1bd6972d852b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:32.930759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 603/1218] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:59:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:32.937046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:33.033217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c9f70-826f-4e0a-9e09-6b85ed704f73 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:33.065066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08724114-e167-4b1e-a243-e8a074639ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:33.065706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08724114-e167-4b1e-a243-e8a074639ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:33.065977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08724114-e167-4b1e-a243-e8a074639ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:33.066247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08724114-e167-4b1e-a243-e8a074639ec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:33.067749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08724114-e167-4b1e-a243-e8a074639ec4 None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=6c36906c508f41cca018e5f6dcaadd91) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:33.068055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08724114-e167-4b1e-a243-e8a074639ec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:33.069462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08724114-e167-4b1e-a243-e8a074639ec4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:33.074925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 616/1219] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:59:32 2022] GET /identity/v3/role_assignments?user.id=6c36906c508f41cca018e5f6dcaadd91&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:35.230031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf7d75-d1e7-4922-9d9c-b531798454fa None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:35.230566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf7d75-d1e7-4922-9d9c-b531798454fa None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:35.230566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf7d75-d1e7-4922-9d9c-b531798454fa None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:35.231967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 604/1220] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:35.237108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceb5b8-fe61-48c8-94ee-eae18a6e2a50 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:35.237778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceb5b8-fe61-48c8-94ee-eae18a6e2a50 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:35.238186 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceb5b8-fe61-48c8-94ee-eae18a6e2a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:35.254953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-83ceb5b8-fe61-48c8-94ee-eae18a6e2a50 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:35.256164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-83ceb5b8-fe61-48c8-94ee-eae18a6e2a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:35.321310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 617/1221] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:35.344287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf7d75-d1e7-4922-9d9c-b531798454fa None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:35.442962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf7d75-d1e7-4922-9d9c-b531798454fa None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:35.476699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:35.477491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:35.477776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:35.478052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:35.487712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:59:35.495378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:35.496560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:35.497667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 605/1222] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:59:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:35.502575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ceb5b8-fe61-48c8-94ee-eae18a6e2a50 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:35.600205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ceb5b8-fe61-48c8-94ee-eae18a6e2a50 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:35.633481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:35.634149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:35.634433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:35.634707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:35.635449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:35.635782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:35.637139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:35.645782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 618/1223] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:59:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:35.651520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:35.748000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcf421-294a-4651-a190-cfb8024becfc None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:35.780965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:35.781748 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:35.782028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:35.782307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:35.792982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:35.794411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:35.811337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:35.812396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:35.813503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:35.813503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:35.813503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:35.813503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:35.813503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 606/1224] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:35.819182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:35.915136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95889784-3514-45e7-b6e4-bb73e5a44aab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:35.948198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:35.949113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:35.949394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:35.949669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:35.950380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:35.950683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:35.952079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:35.963980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 619/1225] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:35.969695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:36.065217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c598c-d5b8-4134-a1fd-9f0f22540b04 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:36.098006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:36.098787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:36.099064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:36.099335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] PATH_INFO: `/v3/users/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:36.106519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:36.108013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:59:36.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:36.123528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:36.124560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 607/1226] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:36.129898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:36.226789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a5cb2-386e-4176-8d57-c76a8356c2a7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:36.259584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:36.260278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:36.260562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:36.260838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:36.261667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:36.261974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:36.263406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:36.279676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 620/1227] 10.222.0.71 () {56 vars in 1143 bytes} [Mon May 16 14:59:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:36.285998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:36.383377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d50e5-bc67-4ddc-af8e-f5ec0d8bc2de None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:36.414662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:36.415520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:36.415825 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:36.416095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:36.427936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:36.429390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:36.445774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:36.446751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:36.447990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:36.447990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:36.447990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:36.447990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:36.447990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 608/1228] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:36.452834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:36.548647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2b017-5b05-4739-b67c-1916abb2a37e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:36.582471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:36.583180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:36.583464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:36.583798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:36.584521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:36.584829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:36.586193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:36.598012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 621/1229] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:36.603782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:36.699495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7aa53-2062-4ba0-a5e1-2c5af721f92c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:36.732161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:36.732931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:36.733207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:36.733480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:36.740110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:36.741556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:59:36.756205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:36.757326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:36.758465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 609/1230] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:59:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:36.763380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:36.859774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bb556-ef8c-4e67-b578-74cf71ce1c1b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:36.892226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0b358-dedd-4483-b5b7-307bbf52a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:36.892886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0b358-dedd-4483-b5b7-307bbf52a953 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:36.894121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0b358-dedd-4483-b5b7-307bbf52a953 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:36.894121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0b358-dedd-4483-b5b7-307bbf52a953 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:36.894808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0b358-dedd-4483-b5b7-307bbf52a953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:36.895118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0b358-dedd-4483-b5b7-307bbf52a953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:36.896566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0b358-dedd-4483-b5b7-307bbf52a953 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:36.908037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 622/1231] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:59:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:36.914726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:37.010831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030c62d-c03c-44db-9496-0812006fc175 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:37.043731 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d153f-2c64-45e6-809a-e9b0e4a87e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:37.044609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d153f-2c64-45e6-809a-e9b0e4a87e84 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:37.044889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d153f-2c64-45e6-809a-e9b0e4a87e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:37.045159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d153f-2c64-45e6-809a-e9b0e4a87e84 None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/6c36906c508f41cca018e5f6dcaadd91/roles/29b60e8912a74b2dafcb91fbb14ee519` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:37.058662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d153f-2c64-45e6-809a-e9b0e4a87e84 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=6c36906c508f41cca018e5f6dcaadd91, role_id=29b60e8912a74b2dafcb91fbb14ee519)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:37.060318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d153f-2c64-45e6-809a-e9b0e4a87e84 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:37.169050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 610/1232] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:59:36 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/6c36906c508f41cca018e5f6dcaadd91/roles/29b60e8912a74b2dafcb91fbb14ee519 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:59:39.373937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d11dd-f966-4ac3-9f2e-2fc4b67e7554 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:39.374499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d11dd-f966-4ac3-9f2e-2fc4b67e7554 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:39.374499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d11dd-f966-4ac3-9f2e-2fc4b67e7554 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:39.375888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 623/1233] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 14:59:39.381797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf501a2-74c5-497e-9615-63a085627e4c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:39.382234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf501a2-74c5-497e-9615-63a085627e4c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:39.382450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf501a2-74c5-497e-9615-63a085627e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:39.399253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-eaf501a2-74c5-497e-9615-63a085627e4c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:39.400509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-eaf501a2-74c5-497e-9615-63a085627e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:39.472139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 611/1234] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:39.494507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d11dd-f966-4ac3-9f2e-2fc4b67e7554 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:39.593233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d11dd-f966-4ac3-9f2e-2fc4b67e7554 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:39.625345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:39.626068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:39.626344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:39.626610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:39.637057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:59:39.644995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:39.646124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:39.647395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 624/1235] 10.222.0.71 () {56 vars in 1102 bytes} [Mon May 16 14:59:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:39.652217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf501a2-74c5-497e-9615-63a085627e4c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:39.745774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf501a2-74c5-497e-9615-63a085627e4c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:39.777999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:39.778660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:39.778943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:39.779217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:39.780052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:39.780355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:39.781723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:39.789696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 612/1236] 10.222.0.71 () {56 vars in 1103 bytes} [Mon May 16 14:59:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:39.795272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:39.893683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b0033-8e0d-4e06-8176-455d12390720 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:39.925404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:39.926181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:39.926459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:39.926730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:39.936647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:39.938076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:39.955045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:39.956156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:39.957103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:39.957103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:39.957103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:39.957103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:39.957103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 625/1237] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:39.962133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:40.057883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472a00a-2ac4-4539-9f58-a2384b3bad36 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:40.089568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:40.090208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:40.090506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:40.090760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:40.091462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:40.091792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:40.093140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:40.104655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 613/1238] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:40.110551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:40.204473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75ec50-77e5-480b-95ed-a1c14d804f90 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:40.238001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:40.238785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:40.239062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:40.239332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:40.246641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:40.248206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:59:40.262175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:40.263301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:40.264526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 626/1239] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:40.269209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:40.363124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf7846-7531-43ff-8908-a8f57bd275d2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:40.395443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:40.396125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:40.396400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:40.396667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:40.397490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:40.397793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:40.399239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:40.416321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 614/1240] 10.222.0.71 () {56 vars in 1143 bytes} [Mon May 16 14:59:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:40.422086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:40.516294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883e183-b060-4a2f-a33b-4299829299fb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:40.547220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:40.547898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:40.548182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:40.548456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:40.613180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:40.614624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:40.709990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 627/1241] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:59:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:59:40.715751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:40.812041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ef588-f3ff-4387-9930-d226164a71d3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:40.845925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:40.846798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:40.847097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:40.847368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:40.857240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:40.858759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:40.876573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:40.877797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:40.878773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:40.878773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:40.878773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:40.878773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:40.878773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 615/1242] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:40.883893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:40.979050 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf453bc-c681-4d41-8c4c-b01bfab368f8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:41.012466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:41.013130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:41.013414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:41.013694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:41.014419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:41.014724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:41.016368 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:41.027399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 628/1243] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:41.033919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:41.131197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3094edb-91a2-4dcf-a50b-77ccf3dd7602 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:41.163450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:41.164247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:41.164561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:41.164831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:41.171249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:41.172710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:59:41.187244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:41.188342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:41.190151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 616/1244] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:41.195130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:41.295380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff30f4-7cac-4754-b9d4-ce9e32665e34 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:41.329133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f9748-546e-4b8a-be30-c23f81f621fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:41.329956 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9748-546e-4b8a-be30-c23f81f621fc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:41.330249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9748-546e-4b8a-be30-c23f81f621fc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:41.330493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9748-546e-4b8a-be30-c23f81f621fc None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:41.331274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9748-546e-4b8a-be30-c23f81f621fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:41.331580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9748-546e-4b8a-be30-c23f81f621fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:41.333041 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9748-546e-4b8a-be30-c23f81f621fc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:41.344646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 629/1245] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:59:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:41.350999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:41.446971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf280-5754-456c-9252-9be54d4d64ba None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:41.481092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259466fa-2f93-4893-a99d-a6aaa9b5a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:41.481748 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259466fa-2f93-4893-a99d-a6aaa9b5a7be None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:41.482037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259466fa-2f93-4893-a99d-a6aaa9b5a7be None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:41.482310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259466fa-2f93-4893-a99d-a6aaa9b5a7be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:41.483135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259466fa-2f93-4893-a99d-a6aaa9b5a7be None admin] RBAC: Adding query filter params (role.id=29b60e8912a74b2dafcb91fbb14ee519, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=6c36906c508f41cca018e5f6dcaadd91) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:41.483445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259466fa-2f93-4893-a99d-a6aaa9b5a7be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:41.484971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259466fa-2f93-4893-a99d-a6aaa9b5a7be None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:41.491372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 617/1246] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:59:41 2022] GET /identity/v3/role_assignments?user.id=6c36906c508f41cca018e5f6dcaadd91&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=29b60e8912a74b2dafcb91fbb14ee519 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:43.778105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e015c-7ef4-495f-a51e-966e6b70c302 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:43.778648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e015c-7ef4-495f-a51e-966e6b70c302 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:43.778648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e015c-7ef4-495f-a51e-966e6b70c302 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:43.780092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 630/1247] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:43.786028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff33af-53d8-4763-b7c9-d0d030fe091e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:43.786470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff33af-53d8-4763-b7c9-d0d030fe091e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:43.786564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff33af-53d8-4763-b7c9-d0d030fe091e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:43.803095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-fdff33af-53d8-4763-b7c9-d0d030fe091e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:43.804301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fdff33af-53d8-4763-b7c9-d0d030fe091e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:43.868861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 618/1248] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:43.891977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e015c-7ef4-495f-a51e-966e6b70c302 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:43.988746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e015c-7ef4-495f-a51e-966e6b70c302 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:44.020670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:44.021453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:44.021731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:44.022000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:44.031729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:59:44.040039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:44.045423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:44.046544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 631/1249] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:59:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:44.047416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff33af-53d8-4763-b7c9-d0d030fe091e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:44.140849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff33af-53d8-4763-b7c9-d0d030fe091e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:44.173994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:44.174661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:44.174941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:44.175213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:44.175998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:44.176318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:44.177652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:44.185430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 619/1250] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:59:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:44.191350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:44.288464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d930ce-8ab9-4bcc-a26f-154abe0688c4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:44.320565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:44.321353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:44.321900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:44.322172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:44.332920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:44.334358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:44.351441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:44.352489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:44.353670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:44.353670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:44.353670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:44.353670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:44.353670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 632/1251] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:44.358442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:44.457159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30d51b-6005-4d43-a753-e36d9e048adc None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:44.489512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:44.490174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:44.490458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:44.490735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:44.491450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:44.491791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:44.493205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:44.505873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 620/1252] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:44.511375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:44.607540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6a9af-0954-4d1d-b927-c5bae80c9d02 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:44.640519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:44.641942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:44.642213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:44.642484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:44.649823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:44.651251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:59:44.665464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:44.666631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:44.668035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 633/1253] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:44.672628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:44.770491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313856c-48c0-4a53-a653-b97ed4e0a3b1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:44.802500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:44.803158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:44.803441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:44.803741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:44.804578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:44.804874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:44.806293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:44.824246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 621/1254] 10.222.0.71 () {56 vars in 1143 bytes} [Mon May 16 14:59:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:44.830787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:44.926960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffd95-b212-4869-9131-d40e5e03aa13 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:44.959117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:44.959778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:44.960057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:44.960325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:45.024579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:45.026008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:45.118101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 634/1255] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:59:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:59:45.124100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:45.222162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5727-5aee-42d0-b991-5e131dd7d775 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:45.256959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:45.257743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:45.258020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:45.258288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:45.268059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:45.269495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:45.286966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:45.287917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:45.288883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:45.288883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:45.288883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:45.288883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:45.288883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 622/1256] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:45.293919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:45.390790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b426041-d8e7-4457-a817-6615617e99f8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:45.423367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:45.424710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:45.425007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:45.425286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:45.426000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:45.426307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:45.427721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:45.438877 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 635/1257] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:45.445093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:45.540961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c4b3-3023-45ea-a1d3-3980f3ac229d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:45.573788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:45.574564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:45.574842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:45.575109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:45.582398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:45.585044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:59:45.597924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:45.598960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:45.600185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 623/1258] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:45.605431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:45.702385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030d9b-6605-4a69-9512-bf706c2e4eb5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:45.735427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214250b-d4a5-427a-a943-539da627226a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:45.736115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214250b-d4a5-427a-a943-539da627226a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:45.736418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214250b-d4a5-427a-a943-539da627226a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:45.736733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214250b-d4a5-427a-a943-539da627226a None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:45.737526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214250b-d4a5-427a-a943-539da627226a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:45.737829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214250b-d4a5-427a-a943-539da627226a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:45.739253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214250b-d4a5-427a-a943-539da627226a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:45.750998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 636/1259] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:59:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:45.757190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:45.852309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d667ca-38ff-4a3d-8a77-dda537358746 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:45.884643 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472646fa-6d08-4c1f-986c-d79ce2aa8523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:45.885299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472646fa-6d08-4c1f-986c-d79ce2aa8523 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:45.885582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472646fa-6d08-4c1f-986c-d79ce2aa8523 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:45.885859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472646fa-6d08-4c1f-986c-d79ce2aa8523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:45.886683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472646fa-6d08-4c1f-986c-d79ce2aa8523 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=6c36906c508f41cca018e5f6dcaadd91) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:45.886995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472646fa-6d08-4c1f-986c-d79ce2aa8523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:45.888453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472646fa-6d08-4c1f-986c-d79ce2aa8523 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:45.894321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 624/1260] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:59:45 2022] GET /identity/v3/role_assignments?user.id=6c36906c508f41cca018e5f6dcaadd91&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=1b3a0717a878438ea224dec0c7639471 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:48.054220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0e439-cdc7-4041-9a2d-2bf7a35d99be None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:48.054842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0e439-cdc7-4041-9a2d-2bf7a35d99be None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:48.054842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0e439-cdc7-4041-9a2d-2bf7a35d99be None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:48.056150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 637/1261] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:48.061172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef869-d2be-435d-ac67-2fcf3ce17c61 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:48.061544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef869-d2be-435d-ac67-2fcf3ce17c61 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:48.061725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef869-d2be-435d-ac67-2fcf3ce17c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:48.079002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f38ef869-d2be-435d-ac67-2fcf3ce17c61 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:48.080715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f38ef869-d2be-435d-ac67-2fcf3ce17c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:48.144685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 625/1262] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:48.166621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0e439-cdc7-4041-9a2d-2bf7a35d99be None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:48.261771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0e439-cdc7-4041-9a2d-2bf7a35d99be None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:48.294337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:48.295129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:48.295412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:48.295712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:48.306082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:59:48.313641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:48.314937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:48.316194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 638/1263] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:48.321150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ef869-d2be-435d-ac67-2fcf3ce17c61 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:48.414646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ef869-d2be-435d-ac67-2fcf3ce17c61 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:48.447023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:48.447706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:48.447992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:48.448269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:48.449008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:48.449309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:48.450671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:48.458389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 626/1264] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:48.464224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:48.564628 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd720d89-8a21-4031-b87f-1326a2bc8524 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:48.597506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:48.598295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:48.598579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:48.598854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:48.608916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:48.611777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:48.627212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:48.628320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:48.629433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:48.629433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:48.629433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:48.629433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:48.629433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 639/1265] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:48.634128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:48.731306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8dd5c-5e4f-46fe-ad01-760aecf346b9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:48.762400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:48.763051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:48.763332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:48.763607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:48.764345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:48.764650 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:48.766043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:48.778647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 627/1266] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:48.785085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:48.883819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178697c4-b22b-436a-8bb0-15461b824f9e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:48.917066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235b605-850e-4439-a183-98afef491e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:48.917855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235b605-850e-4439-a183-98afef491e59 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:48.918137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235b605-850e-4439-a183-98afef491e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:48.918414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235b605-850e-4439-a183-98afef491e59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:48.926059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235b605-850e-4439-a183-98afef491e59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:48.927502 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235b605-850e-4439-a183-98afef491e59 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6235b605-850e-4439-a183-98afef491e59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:59:48.941924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:48.944913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:48.946364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 640/1267] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:48.949267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:49.047103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd5a22-fe30-4cff-ad66-c0cd95e9291c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:49.079182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:49.080963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:49.081244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:49.081517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:49.082343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:49.082642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:49.084099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:49.100139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 628/1268] 10.222.0.71 () {56 vars in 1143 bytes} [Mon May 16 14:59:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:49.105961 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235b605-850e-4439-a183-98afef491e59 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:49.202604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235b605-850e-4439-a183-98afef491e59 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:49.235106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:49.236080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:49.236361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:49.236632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:49.246670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:49.248145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:49.265065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:49.266158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:49.267865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:49.267865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:49.267865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:49.267865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:49.267865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 641/1269] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:49.272794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:49.368698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c22c3e-a5d3-4970-9d10-bab3580beea7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:49.401316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:49.401972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:49.402253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:49.402525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:49.403241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:49.403546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:49.404963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:49.415601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 629/1270] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:49.421422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:49.520453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5eb0d4-22e0-40fa-92be-f93ccc9fb762 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:49.552449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:49.553287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:49.553565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:49.553836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:49.560178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:49.561608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:59:49.575890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:49.582642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:49.583770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 642/1271] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:49.584367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:49.679713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066142e4-7df1-4a64-a3db-50a71529bfab None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:49.711170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a931c-f80b-4d78-95ae-a6cda1152adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:49.711846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a931c-f80b-4d78-95ae-a6cda1152adb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:49.712134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a931c-f80b-4d78-95ae-a6cda1152adb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:49.712413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a931c-f80b-4d78-95ae-a6cda1152adb None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:49.713243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a931c-f80b-4d78-95ae-a6cda1152adb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:49.713554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a931c-f80b-4d78-95ae-a6cda1152adb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:49.714971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a931c-f80b-4d78-95ae-a6cda1152adb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:49.727354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 630/1272] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:49.733086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:49.829907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e201-d82d-4dac-83c2-ae68f17a7d2d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:49.864655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1153d35-a70e-45aa-981d-811ce8b5646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:49.865540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1153d35-a70e-45aa-981d-811ce8b5646b None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:49.865837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1153d35-a70e-45aa-981d-811ce8b5646b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:49.866112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1153d35-a70e-45aa-981d-811ce8b5646b None admin] PATH_INFO: `/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/6c36906c508f41cca018e5f6dcaadd91/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:49.868224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1153d35-a70e-45aa-981d-811ce8b5646b None admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95ef9b6a74897922ac44c55f68e3b, user_id=6c36906c508f41cca018e5f6dcaadd91, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:49.869659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1153d35-a70e-45aa-981d-811ce8b5646b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:49.943247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 643/1273] 10.222.0.71 () {58 vars in 1439 bytes} [Mon May 16 14:59:49 2022] PUT /identity/v3/projects/8ef95ef9b6a74897922ac44c55f68e3b/users/6c36906c508f41cca018e5f6dcaadd91/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 14:59:52.161599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68302e3-acf1-4483-b8ec-f9c243a326ec None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:52.162099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68302e3-acf1-4483-b8ec-f9c243a326ec None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:52.162099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68302e3-acf1-4483-b8ec-f9c243a326ec None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:52.163438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 631/1274] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:52.168778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf6d1-2761-42d3-85b3-cce7612b7e81 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:52.169161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf6d1-2761-42d3-85b3-cce7612b7e81 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:52.169324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf6d1-2761-42d3-85b3-cce7612b7e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:52.185705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-52abf6d1-2761-42d3-85b3-cce7612b7e81 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:52.186926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-52abf6d1-2761-42d3-85b3-cce7612b7e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:52.252824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 644/1275] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:52.276218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68302e3-acf1-4483-b8ec-f9c243a326ec None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:52.372516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68302e3-acf1-4483-b8ec-f9c243a326ec None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:52.404873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:52.405656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:52.405939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:52.406216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:52.416504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 14:59:52.424790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:52.425872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:52.426812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 632/1276] 10.222.0.71 () {56 vars in 1096 bytes} [Mon May 16 14:59:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:52.431917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf6d1-2761-42d3-85b3-cce7612b7e81 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:52.527828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf6d1-2761-42d3-85b3-cce7612b7e81 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:52.559603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:52.560292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:52.560579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:52.560853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:52.561589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:52.561922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:52.564143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:52.572641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 645/1277] 10.222.0.71 () {56 vars in 1099 bytes} [Mon May 16 14:59:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:52.578381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:52.675712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bf7b0-a20d-4c89-9c16-7388e47ddc2a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:52.708310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:52.709087 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:52.709368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:52.709642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:52.719643 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:52.721084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:52.738618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:52.739718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:52.740790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:52.740790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:52.740790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:52.740790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:52.740790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 633/1278] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:52.746103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:52.840425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343502a-665c-4402-bb27-4ea7a7a3c4b1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:52.874254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:52.874912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:52.875195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:52.875565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:52.876637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:52.876943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:52.878296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:52.889115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 646/1279] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:52.895104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:52.988882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997c7b8-247a-492f-8638-f20222018077 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:53.020259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:53.021086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:53.021358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:53.021623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:53.028479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:53.029933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 14:59:53.044536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:53.045597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:53.046721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 634/1280] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:53.052559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:53.148194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef049c-b6b5-46b9-b6d5-bf5ffcf2cd6f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:53.180703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:53.181370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:53.181739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:53.181914 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:53.182744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:53.183049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:53.184494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:53.201755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 647/1281] 10.222.0.71 () {56 vars in 1143 bytes} [Mon May 16 14:59:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:53.207976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:53.301179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512631-ffae-4262-a9bc-35da57b4bf0f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:53.333053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:53.333678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:53.333956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:53.334224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:53.396932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:53.398357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:53.491921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 635/1282] 10.222.0.71 () {58 vars in 1282 bytes} [Mon May 16 14:59:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 14:59:53.497638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:53.597882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2f24c-548b-4a60-aacb-cb747dd6fe9b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:53.630638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:53.631455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:53.631768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:53.632130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:53.643003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:53.644626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 14:59:53.663467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:53.664606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:53.671752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 14:59:53.671752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 14:59:53.671752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 14:59:53.671752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:53.671752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 648/1283] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 14:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:53.672120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:53.769564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47522638-eb17-4ba3-af67-27e7eaf005cf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:53.806066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:53.806784 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:53.807061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:53.807331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:53.808099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:53.808400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:53.809812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:53.822652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 636/1284] 10.222.0.71 () {56 vars in 1109 bytes} [Mon May 16 14:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:53.828708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:53.933572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8279da-61c2-4b61-be65-275dea8462c9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:53.968924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:53.969853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:53.970137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:53.970422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:53.978071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:53.979530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 14:59:53.994558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:53.995802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 16 14:59:53.996934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 649/1285] 10.222.0.71 () {56 vars in 1111 bytes} [Mon May 16 14:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 14:59:54.004481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:54.111426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744b55-03fc-4214-b508-3171fcd07e19 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:54.146056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1e64-cfc1-4f3c-922b-df8803aed200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:54.146812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1e64-cfc1-4f3c-922b-df8803aed200 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:54.147093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1e64-cfc1-4f3c-922b-df8803aed200 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:54.147380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1e64-cfc1-4f3c-922b-df8803aed200 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:54.148199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1e64-cfc1-4f3c-922b-df8803aed200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:54.148504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1e64-cfc1-4f3c-922b-df8803aed200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:54.149941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1e64-cfc1-4f3c-922b-df8803aed200 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:54.162535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 637/1286] 10.222.0.71 () {56 vars in 1148 bytes} [Mon May 16 14:59:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:54.168565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:54.270346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f82e0-2417-4317-9185-e34a6a09f8eb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:54.303200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f22a4f-e7ff-49dc-989c-dc29f63da0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:54.303884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22a4f-e7ff-49dc-989c-dc29f63da0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:54.304167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22a4f-e7ff-49dc-989c-dc29f63da0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:54.304440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22a4f-e7ff-49dc-989c-dc29f63da0d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:54.305264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f22a4f-e7ff-49dc-989c-dc29f63da0d3 None admin] RBAC: Adding query filter params (role.id=1b3a0717a878438ea224dec0c7639471, scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b, user.id=6c36906c508f41cca018e5f6dcaadd91) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:54.305573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f22a4f-e7ff-49dc-989c-dc29f63da0d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:54.307133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f22a4f-e7ff-49dc-989c-dc29f63da0d3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:54.313053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 650/1287] 10.222.0.71 () {56 vars in 1374 bytes} [Mon May 16 14:59:54 2022] GET /identity/v3/role_assignments?user.id=6c36906c508f41cca018e5f6dcaadd91&scope.project.id=8ef95ef9b6a74897922ac44c55f68e3b&role.id=1b3a0717a878438ea224dec0c7639471 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:56.514517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59dd33-dda5-4b98-a0e7-84e1e4655707 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:56.515759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59dd33-dda5-4b98-a0e7-84e1e4655707 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:56.515759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59dd33-dda5-4b98-a0e7-84e1e4655707 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:56.517597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 638/1288] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:56.522769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae5c99-ebb3-412d-93f5-b4ab08bd2d42 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:56.523202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae5c99-ebb3-412d-93f5-b4ab08bd2d42 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:56.523365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae5c99-ebb3-412d-93f5-b4ab08bd2d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:56.540196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5eae5c99-ebb3-412d-93f5-b4ab08bd2d42 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:56.541389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5eae5c99-ebb3-412d-93f5-b4ab08bd2d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:56.605309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 651/1289] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:56.627334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59dd33-dda5-4b98-a0e7-84e1e4655707 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:56.723440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59dd33-dda5-4b98-a0e7-84e1e4655707 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:56.756409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:56.757193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:56.757475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:56.757750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:56.758394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:56.759811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 14:59:56.772568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 14:59:56.773824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 14:59:56.774928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 639/1290] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 14:59:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 14:59:56.779803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae5c99-ebb3-412d-93f5-b4ab08bd2d42 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:56.874053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae5c99-ebb3-412d-93f5-b4ab08bd2d42 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:56.907214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a552a2f-add7-4cfc-9148-fb08fe7df5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:56.908014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552a2f-add7-4cfc-9148-fb08fe7df5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:56.908294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552a2f-add7-4cfc-9148-fb08fe7df5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:56.908565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552a2f-add7-4cfc-9148-fb08fe7df5d8 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:56.909256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a552a2f-add7-4cfc-9148-fb08fe7df5d8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:56.909554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a552a2f-add7-4cfc-9148-fb08fe7df5d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:56.910934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a552a2f-add7-4cfc-9148-fb08fe7df5d8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:56.915492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 652/1291] 10.222.0.71 () {56 vars in 1116 bytes} [Mon May 16 14:59:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:56.921814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:57.015038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3247a-3309-4004-81ca-f3f0980d78e5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:57.045955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3704be-d4ff-4137-93b6-f197b9dcac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:57.047052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3704be-d4ff-4137-93b6-f197b9dcac3a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:57.047326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3704be-d4ff-4137-93b6-f197b9dcac3a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:57.047595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3704be-d4ff-4137-93b6-f197b9dcac3a None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:57.048315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3704be-d4ff-4137-93b6-f197b9dcac3a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 14:59:57.048615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3704be-d4ff-4137-93b6-f197b9dcac3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:57.049965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3704be-d4ff-4137-93b6-f197b9dcac3a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:57.054270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 640/1292] 10.222.0.71 () {56 vars in 1116 bytes} [Mon May 16 14:59:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 14:59:59.260744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee847-d88c-4389-92c6-1c2dea0a75b8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:59.261326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee847-d88c-4389-92c6-1c2dea0a75b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:59.261326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee847-d88c-4389-92c6-1c2dea0a75b8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:59.262643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 653/1293] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 14:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 14:59:59.268582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778db3d-4d9e-4b61-82ad-4374550f2f52 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:59.269038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778db3d-4d9e-4b61-82ad-4374550f2f52 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:59.269139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778db3d-4d9e-4b61-82ad-4374550f2f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:59.285892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6778db3d-4d9e-4b61-82ad-4374550f2f52 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 14:59:59.287070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6778db3d-4d9e-4b61-82ad-4374550f2f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 14:59:59.350931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 641/1294] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 14:59:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 14:59:59.373485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee847-d88c-4389-92c6-1c2dea0a75b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 14:59:59.472298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee847-d88c-4389-92c6-1c2dea0a75b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 14:59:59.504925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bec44-d2cb-4506-b734-82df038e9d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 14:59:59.505593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bec44-d2cb-4506-b734-82df038e9d0f None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 14:59:59.505876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bec44-d2cb-4506-b734-82df038e9d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 14:59:59.506153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bec44-d2cb-4506-b734-82df038e9d0f None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 14:59:59.506873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bec44-d2cb-4506-b734-82df038e9d0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 14:59:59.508270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bec44-d2cb-4506-b734-82df038e9d0f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 14:59:59.597922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 654/1295] 10.222.0.71 () {60 vars in 1141 bytes} [Mon May 16 14:59:59 2022] POST /identity/v3/services => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:00:01.784454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd0080-9d6b-4af4-aa13-baceef0bd299 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:01.785010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd0080-9d6b-4af4-aa13-baceef0bd299 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:01.785010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd0080-9d6b-4af4-aa13-baceef0bd299 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:01.786398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 642/1296] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:00:01.791469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd97f70-eb5d-4f81-8a83-76add24726b0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:01.791943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd97f70-eb5d-4f81-8a83-76add24726b0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:01.792058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd97f70-eb5d-4f81-8a83-76add24726b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:01.808754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dfd97f70-eb5d-4f81-8a83-76add24726b0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:00:01.814974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dfd97f70-eb5d-4f81-8a83-76add24726b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:00:01.883210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 655/1297] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:00:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 15:00:01.905635 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd0080-9d6b-4af4-aa13-baceef0bd299 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:00:02.003075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd0080-9d6b-4af4-aa13-baceef0bd299 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:00:02.035681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:00:02.036409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:02.036692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:02.036962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:02.037607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:00:02.039172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 15:00:02.051599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 15:00:02.061743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 15:00:02.063003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 643/1298] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 15:00:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 15:00:02.064057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd97f70-eb5d-4f81-8a83-76add24726b0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:00:02.152475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd97f70-eb5d-4f81-8a83-76add24726b0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:00:02.186364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:00:02.187255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:02.187532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:02.187874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:02.193788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:00:02.193788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:00:02.193788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:00:02.195519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 656/1299] 10.222.0.71 () {56 vars in 1116 bytes} [Mon May 16 15:00:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:00:02.201776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:00:02.298079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9b135-6dbc-435c-8d6a-ebff6f8bf34b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:00:02.330014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5a640-87da-48a7-b4f8-63500ccc5cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:00:02.330651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5a640-87da-48a7-b4f8-63500ccc5cda None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:02.330927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5a640-87da-48a7-b4f8-63500ccc5cda None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:02.331197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5a640-87da-48a7-b4f8-63500ccc5cda None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:02.332005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5a640-87da-48a7-b4f8-63500ccc5cda None admin] RBAC: Adding query filter params (interface=public, service_id=a8e326e3e9994aa482eb6ed94ed33d2d, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:00:02.332307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5a640-87da-48a7-b4f8-63500ccc5cda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:00:02.333701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5a640-87da-48a7-b4f8-63500ccc5cda None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:00:02.338528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 644/1300] 10.222.0.71 () {56 vars in 1251 bytes} [Mon May 16 15:00:02 2022] GET /identity/v3/endpoints?service_id=a8e326e3e9994aa482eb6ed94ed33d2d&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:00:02.344415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:00:02.441770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf05cb-59e5-425c-bb30-eb7b91321032 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:00:02.475036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13f744-1416-4f89-95ad-db3782aa7792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:00:02.475748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13f744-1416-4f89-95ad-db3782aa7792 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:02.476045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13f744-1416-4f89-95ad-db3782aa7792 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:02.476322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13f744-1416-4f89-95ad-db3782aa7792 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:02.476996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13f744-1416-4f89-95ad-db3782aa7792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:00:02.478359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13f744-1416-4f89-95ad-db3782aa7792 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:00:02.485643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 657/1301] 10.222.0.71 () {56 vars in 1087 bytes} [Mon May 16 15:00:02 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:00:04.674725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64a1a2-39a8-41bf-b4f9-433c5b8ec1b1 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:04.675349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64a1a2-39a8-41bf-b4f9-433c5b8ec1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:04.675349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64a1a2-39a8-41bf-b4f9-433c5b8ec1b1 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:04.676656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 645/1302] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:00:04.681776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1adb51-24bf-4042-8f75-fc98db8d9a32 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:04.682156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1adb51-24bf-4042-8f75-fc98db8d9a32 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:04.682325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1adb51-24bf-4042-8f75-fc98db8d9a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:04.699129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5e1adb51-24bf-4042-8f75-fc98db8d9a32 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:00:04.700332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e1adb51-24bf-4042-8f75-fc98db8d9a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:00:04.764618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 658/1303] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:00:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 15:00:04.787972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64a1a2-39a8-41bf-b4f9-433c5b8ec1b1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:00:04.886728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64a1a2-39a8-41bf-b4f9-433c5b8ec1b1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:00:04.918661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:00:04.920110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:04.920395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:04.920664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:04.921300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:00:04.922643 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 15:00:04.933992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 15:00:04.935204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 16 15:00:04.938233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 646/1304] 10.222.0.71 () {56 vars in 1117 bytes} [Mon May 16 15:00:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 15:00:04.941965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1adb51-24bf-4042-8f75-fc98db8d9a32 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:00:05.037950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1adb51-24bf-4042-8f75-fc98db8d9a32 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:00:05.071129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20562293-3715-4fdb-966a-d5108ae84e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:00:05.071805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20562293-3715-4fdb-966a-d5108ae84e4e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:05.072089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20562293-3715-4fdb-966a-d5108ae84e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:05.072365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20562293-3715-4fdb-966a-d5108ae84e4e None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:05.073062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20562293-3715-4fdb-966a-d5108ae84e4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:00:05.073364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20562293-3715-4fdb-966a-d5108ae84e4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:00:05.074722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20562293-3715-4fdb-966a-d5108ae84e4e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:00:05.080718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 659/1305] 10.222.0.71 () {56 vars in 1116 bytes} [Mon May 16 15:00:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:00:05.086414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:00:05.179601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0395a-5e6c-4dbf-92d4-1f969d16fe6d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:00:05.211693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bbdbf-5e93-41ac-8c59-922f9b14c7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:00:05.212348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bbdbf-5e93-41ac-8c59-922f9b14c7ce None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:00:05.212630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bbdbf-5e93-41ac-8c59-922f9b14c7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:00:05.212904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bbdbf-5e93-41ac-8c59-922f9b14c7ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:00:05.213637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bbdbf-5e93-41ac-8c59-922f9b14c7ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:00:05.214994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bbdbf-5e93-41ac-8c59-922f9b14c7ce None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:00:05.333370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 647/1306] 10.222.0.71 () {60 vars in 1144 bytes} [Mon May 16 15:00:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:06:06.195864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079382bc-15b2-49ef-a8b5-993317cfcea9 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:06.196871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079382bc-15b2-49ef-a8b5-993317cfcea9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:06.197387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079382bc-15b2-49ef-a8b5-993317cfcea9 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:06.199219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 660/1307] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:06.204963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bf330-d751-4e25-bc34-892f1f94c2c3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:06.206340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bf330-d751-4e25-bc34-892f1f94c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:06.207231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bf330-d751-4e25-bc34-892f1f94c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:06.306610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-747bf330-d751-4e25-bc34-892f1f94c2c3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:06.307881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-747bf330-d751-4e25-bc34-892f1f94c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:06.431167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 648/1308] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:06.488379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e59e-b234-4801-ab0b-67b9c3c9236e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:06.488609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e59e-b234-4801-ab0b-67b9c3c9236e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:06.489061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e59e-b234-4801-ab0b-67b9c3c9236e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:06.491058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 661/1309] 10.222.0.71 () {54 vars in 964 bytes} [Mon May 16 15:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:06.497253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cef1f-9bd7-41b7-abfb-8d5632550cc8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:06.497549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cef1f-9bd7-41b7-abfb-8d5632550cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:06.497832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cef1f-9bd7-41b7-abfb-8d5632550cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:06.574569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7b7cef1f-9bd7-41b7-abfb-8d5632550cc8 None None] MFA Rules not processed for user `76b2a2d381a14f5fb04e721905b36ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:06.575951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7b7cef1f-9bd7-41b7-abfb-8d5632550cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:06.653002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 649/1310] 10.222.0.71 () {58 vars in 1061 bytes} [Mon May 16 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:06.662788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24e59e-b234-4801-ab0b-67b9c3c9236e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:06.664667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dc24e59e-b234-4801-ab0b-67b9c3c9236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:06.769881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24e59e-b234-4801-ab0b-67b9c3c9236e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:06.841465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658abc52-7320-4cc6-98b8-17a399e5b423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:06.842214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abc52-7320-4cc6-98b8-17a399e5b423 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:06.842498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abc52-7320-4cc6-98b8-17a399e5b423 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:06.842770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abc52-7320-4cc6-98b8-17a399e5b423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:06.844464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-658abc52-7320-4cc6-98b8-17a399e5b423 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:06.952173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658abc52-7320-4cc6-98b8-17a399e5b423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:06.953626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658abc52-7320-4cc6-98b8-17a399e5b423 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:06.971677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 662/1311] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:09.478516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627b6f-bb66-4b0f-896c-a9ec41f77c60 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:09.479072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627b6f-bb66-4b0f-896c-a9ec41f77c60 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:09.479072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627b6f-bb66-4b0f-896c-a9ec41f77c60 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:09.480631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 650/1312] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:09.486443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbe221-b613-4435-b8e0-21c059e87543 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:09.486929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbe221-b613-4435-b8e0-21c059e87543 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:09.487031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbe221-b613-4435-b8e0-21c059e87543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:09.559140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-95fbe221-b613-4435-b8e0-21c059e87543 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:09.560372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95fbe221-b613-4435-b8e0-21c059e87543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:09.656447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 663/1313] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:09.694751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3627b6f-bb66-4b0f-896c-a9ec41f77c60 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:09.696318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c3627b6f-bb66-4b0f-896c-a9ec41f77c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:09.799270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3627b6f-bb66-4b0f-896c-a9ec41f77c60 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:09.871692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5000e5e-af9d-48d3-be1e-1769b5a8abc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:09.872349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000e5e-af9d-48d3-be1e-1769b5a8abc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:09.872630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000e5e-af9d-48d3-be1e-1769b5a8abc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:09.872898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000e5e-af9d-48d3-be1e-1769b5a8abc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:09.874548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d5000e5e-af9d-48d3-be1e-1769b5a8abc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:09.988636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5000e5e-af9d-48d3-be1e-1769b5a8abc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:09.990043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5000e5e-af9d-48d3-be1e-1769b5a8abc6 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:10.008848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 651/1314] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:06:12.821741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7589f7-07d8-4c4f-8683-b731628ed839 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:12.822299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7589f7-07d8-4c4f-8683-b731628ed839 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:12.822299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7589f7-07d8-4c4f-8683-b731628ed839 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:12.823723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 664/1315] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:12.829727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c128e-0da8-4763-98cf-35347ac5a35e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:12.830229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c128e-0da8-4763-98cf-35347ac5a35e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:12.830340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c128e-0da8-4763-98cf-35347ac5a35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:12.901271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b60c128e-0da8-4763-98cf-35347ac5a35e None None] MFA Rules not processed for user `160bfc163c5d49f9a9bae22a11e7f2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:12.902551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b60c128e-0da8-4763-98cf-35347ac5a35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:13.001081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 652/1316] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:13.041355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7589f7-07d8-4c4f-8683-b731628ed839 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:13.042924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a7589f7-07d8-4c4f-8683-b731628ed839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:13.141827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7589f7-07d8-4c4f-8683-b731628ed839 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:13.211171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60c8a6-1061-4ad7-b28b-f0e5332c891a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:13.211852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60c8a6-1061-4ad7-b28b-f0e5332c891a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:13.212134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60c8a6-1061-4ad7-b28b-f0e5332c891a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:13.212405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60c8a6-1061-4ad7-b28b-f0e5332c891a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:13.214064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0b60c8a6-1061-4ad7-b28b-f0e5332c891a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:13.321605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60c8a6-1061-4ad7-b28b-f0e5332c891a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:13.323052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60c8a6-1061-4ad7-b28b-f0e5332c891a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:13.340773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 665/1317] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:16.363645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de96fb-3490-48fd-8823-ef865ab538f8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:16.364318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de96fb-3490-48fd-8823-ef865ab538f8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:16.364318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de96fb-3490-48fd-8823-ef865ab538f8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:16.365691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 653/1318] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:16.370904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b01f-7c2a-49a3-a049-a971ae9e9513 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:16.371321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b01f-7c2a-49a3-a049-a971ae9e9513 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:16.371459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b01f-7c2a-49a3-a049-a971ae9e9513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:16.432976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2dc2b01f-7c2a-49a3-a049-a971ae9e9513 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:16.434150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2dc2b01f-7c2a-49a3-a049-a971ae9e9513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:16.524750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 666/1319] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:16.563513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de96fb-3490-48fd-8823-ef865ab538f8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:16.565046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-76de96fb-3490-48fd-8823-ef865ab538f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:16.663270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de96fb-3490-48fd-8823-ef865ab538f8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:16.733105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e2b93-231c-41f8-a531-5cc164204c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:16.733741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2b93-231c-41f8-a531-5cc164204c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:16.734016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2b93-231c-41f8-a531-5cc164204c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:16.734281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2b93-231c-41f8-a531-5cc164204c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:16.735930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-380e2b93-231c-41f8-a531-5cc164204c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:16.839093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e2b93-231c-41f8-a531-5cc164204c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:16.842639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e2b93-231c-41f8-a531-5cc164204c71 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:16.858730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 654/1320] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:20.047360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b971-6cfe-4c64-9f21-5501e7098bc2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:20.047916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b971-6cfe-4c64-9f21-5501e7098bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:20.048030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b971-6cfe-4c64-9f21-5501e7098bc2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:20.049318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 667/1321] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:20.055088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98580230-3eb1-43ca-8e6b-8441b2fa4266 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:20.055531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98580230-3eb1-43ca-8e6b-8441b2fa4266 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:20.055671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98580230-3eb1-43ca-8e6b-8441b2fa4266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:20.118310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-98580230-3eb1-43ca-8e6b-8441b2fa4266 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:20.119435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-98580230-3eb1-43ca-8e6b-8441b2fa4266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:20.213825 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 655/1322] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:06:20.248580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b971-6cfe-4c64-9f21-5501e7098bc2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:20.250189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2392b971-6cfe-4c64-9f21-5501e7098bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:20.349271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b971-6cfe-4c64-9f21-5501e7098bc2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:20.419024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cf1ce-44fe-479e-92f0-ba9d69c756a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:20.419591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf1ce-44fe-479e-92f0-ba9d69c756a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:20.419896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf1ce-44fe-479e-92f0-ba9d69c756a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:20.420168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf1ce-44fe-479e-92f0-ba9d69c756a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:20.421777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c60cf1ce-44fe-479e-92f0-ba9d69c756a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:20.525550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cf1ce-44fe-479e-92f0-ba9d69c756a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:20.526978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cf1ce-44fe-479e-92f0-ba9d69c756a0 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:20.546018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 668/1323] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:22.840661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cbac4-f312-4862-b7bf-d26d48be1754 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:22.841190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cbac4-f312-4862-b7bf-d26d48be1754 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:22.841190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cbac4-f312-4862-b7bf-d26d48be1754 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:22.843674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 656/1324] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:22.851787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612dbf4-c867-47c1-b459-a17d4ddc92ee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:22.851787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612dbf4-c867-47c1-b459-a17d4ddc92ee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:22.851787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612dbf4-c867-47c1-b459-a17d4ddc92ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:22.917545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9612dbf4-c867-47c1-b459-a17d4ddc92ee None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:22.918740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9612dbf4-c867-47c1-b459-a17d4ddc92ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:23.013410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 669/1325] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:23.048121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cbac4-f312-4862-b7bf-d26d48be1754 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:23.049674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a94cbac4-f312-4862-b7bf-d26d48be1754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:23.149243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cbac4-f312-4862-b7bf-d26d48be1754 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:23.219168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749548a6-666a-4739-bb48-a77b94cf5980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:23.219843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749548a6-666a-4739-bb48-a77b94cf5980 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:23.220129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749548a6-666a-4739-bb48-a77b94cf5980 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:23.220398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749548a6-666a-4739-bb48-a77b94cf5980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:23.222018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-749548a6-666a-4739-bb48-a77b94cf5980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:23.325866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749548a6-666a-4739-bb48-a77b94cf5980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:23.327301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749548a6-666a-4739-bb48-a77b94cf5980 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:23.345837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 657/1326] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:25.659622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562cd7d-0da3-4aaa-8307-7441780ac933 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:25.661045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562cd7d-0da3-4aaa-8307-7441780ac933 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:25.661335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562cd7d-0da3-4aaa-8307-7441780ac933 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:25.662717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 670/1327] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:25.668089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aa513-8943-4bb0-846d-d74c3ffdcb87 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:25.668479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aa513-8943-4bb0-846d-d74c3ffdcb87 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:25.668638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aa513-8943-4bb0-846d-d74c3ffdcb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:25.741932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-976aa513-8943-4bb0-846d-d74c3ffdcb87 None None] MFA Rules not processed for user `160bfc163c5d49f9a9bae22a11e7f2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:25.743141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-976aa513-8943-4bb0-846d-d74c3ffdcb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:25.836239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 658/1328] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:25.874456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562cd7d-0da3-4aaa-8307-7441780ac933 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:25.876057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c562cd7d-0da3-4aaa-8307-7441780ac933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:25.980804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562cd7d-0da3-4aaa-8307-7441780ac933 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:26.052859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ada0e7-0abf-40bd-b601-17cfcc93887b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:26.053519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ada0e7-0abf-40bd-b601-17cfcc93887b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:26.053800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ada0e7-0abf-40bd-b601-17cfcc93887b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:26.054077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ada0e7-0abf-40bd-b601-17cfcc93887b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:26.055734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-64ada0e7-0abf-40bd-b601-17cfcc93887b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:26.162141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ada0e7-0abf-40bd-b601-17cfcc93887b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:26.163585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ada0e7-0abf-40bd-b601-17cfcc93887b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:26.181508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 671/1329] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:28.849936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a960-e171-415b-b7e5-68b6c5da6708 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:28.850388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a960-e171-415b-b7e5-68b6c5da6708 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:28.850487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a960-e171-415b-b7e5-68b6c5da6708 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:28.851943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 659/1330] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:28.857334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd1e1-d86d-4906-a382-e6f5047cea66 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:28.857720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd1e1-d86d-4906-a382-e6f5047cea66 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:28.857892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd1e1-d86d-4906-a382-e6f5047cea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:28.924460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dcbdd1e1-d86d-4906-a382-e6f5047cea66 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:28.926259 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dcbdd1e1-d86d-4906-a382-e6f5047cea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:29.020794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 672/1331] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:29.057139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a960-e171-415b-b7e5-68b6c5da6708 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:29.058723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2196a960-e171-415b-b7e5-68b6c5da6708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:29.156509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a960-e171-415b-b7e5-68b6c5da6708 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:29.226591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4447472-d976-4857-81e6-3998c1a417fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:29.227228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4447472-d976-4857-81e6-3998c1a417fc service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:29.227504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4447472-d976-4857-81e6-3998c1a417fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:29.227803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4447472-d976-4857-81e6-3998c1a417fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:29.229410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e4447472-d976-4857-81e6-3998c1a417fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:29.340713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4447472-d976-4857-81e6-3998c1a417fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:29.342177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4447472-d976-4857-81e6-3998c1a417fc service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:29.362375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 660/1332] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:31.679684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbff05d-c4af-4a0c-8d01-49bdad887d6c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:31.680189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbff05d-c4af-4a0c-8d01-49bdad887d6c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:31.680189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbff05d-c4af-4a0c-8d01-49bdad887d6c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:31.681560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 673/1333] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:31.686755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59c579-5b7f-44a1-9974-11ff0d194da4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:31.687139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59c579-5b7f-44a1-9974-11ff0d194da4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:31.687305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59c579-5b7f-44a1-9974-11ff0d194da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:31.749497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2a59c579-5b7f-44a1-9974-11ff0d194da4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:31.750692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2a59c579-5b7f-44a1-9974-11ff0d194da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:31.841891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 661/1334] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:31.881424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbff05d-c4af-4a0c-8d01-49bdad887d6c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:31.882950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5dbff05d-c4af-4a0c-8d01-49bdad887d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:31.982463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbff05d-c4af-4a0c-8d01-49bdad887d6c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:32.053761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22329a-75a5-4d69-b711-0477a0c4224a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:32.054411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22329a-75a5-4d69-b711-0477a0c4224a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:32.054694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22329a-75a5-4d69-b711-0477a0c4224a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:32.054970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22329a-75a5-4d69-b711-0477a0c4224a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:32.056636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fe22329a-75a5-4d69-b711-0477a0c4224a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:32.161639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22329a-75a5-4d69-b711-0477a0c4224a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:32.163076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22329a-75a5-4d69-b711-0477a0c4224a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:32.181031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 674/1335] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:34.986628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64c93f-2988-4a46-9843-90caa252cb9b None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:34.987240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64c93f-2988-4a46-9843-90caa252cb9b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:34.987240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64c93f-2988-4a46-9843-90caa252cb9b None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:34.989141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 662/1336] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:34.994381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58769f7-9061-40ed-8307-c1cc8c54a8a6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:34.994834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58769f7-9061-40ed-8307-c1cc8c54a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:34.994948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58769f7-9061-40ed-8307-c1cc8c54a8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:35.058470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e58769f7-9061-40ed-8307-c1cc8c54a8a6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:35.059657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e58769f7-9061-40ed-8307-c1cc8c54a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:35.153305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 675/1337] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:35.188927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64c93f-2988-4a46-9843-90caa252cb9b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:35.190469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7b64c93f-2988-4a46-9843-90caa252cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:35.290891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64c93f-2988-4a46-9843-90caa252cb9b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:35.360668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d8ed6-3e93-4879-b4c4-ab5ac583ca81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:35.361309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d8ed6-3e93-4879-b4c4-ab5ac583ca81 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:35.361590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d8ed6-3e93-4879-b4c4-ab5ac583ca81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:35.361865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d8ed6-3e93-4879-b4c4-ab5ac583ca81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:35.364490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-658d8ed6-3e93-4879-b4c4-ab5ac583ca81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:35.470537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d8ed6-3e93-4879-b4c4-ab5ac583ca81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:35.472038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d8ed6-3e93-4879-b4c4-ab5ac583ca81 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:35.491804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 663/1338] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:39.865746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ae47-5cfb-495f-aff7-07e9ff04ae72 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:39.866261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ae47-5cfb-495f-aff7-07e9ff04ae72 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:39.866344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ae47-5cfb-495f-aff7-07e9ff04ae72 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:39.867731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 676/1339] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:39.873326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26c329-0d3a-465c-8e57-79659d7215df None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:39.873787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26c329-0d3a-465c-8e57-79659d7215df None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:39.873897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26c329-0d3a-465c-8e57-79659d7215df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:39.944923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6d26c329-0d3a-465c-8e57-79659d7215df None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:39.946056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6d26c329-0d3a-465c-8e57-79659d7215df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:40.172696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 664/1340] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:40.210860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ae47-5cfb-495f-aff7-07e9ff04ae72 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:40.213055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-82e6ae47-5cfb-495f-aff7-07e9ff04ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:40.307007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ae47-5cfb-495f-aff7-07e9ff04ae72 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:40.310582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6ee32-311a-4b61-ad7f-aa2e0a2ef582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:40.311231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ee32-311a-4b61-ad7f-aa2e0a2ef582 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:40.311513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ee32-311a-4b61-ad7f-aa2e0a2ef582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:40.311822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6ee32-311a-4b61-ad7f-aa2e0a2ef582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:40.321351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6ee32-311a-4b61-ad7f-aa2e0a2ef582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:40.322785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6ee32-311a-4b61-ad7f-aa2e0a2ef582 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:40.332103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 677/1341] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:43.475089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48ca4-c9ee-4575-a948-129cf1dd814c None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:43.475622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48ca4-c9ee-4575-a948-129cf1dd814c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:43.475622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db48ca4-c9ee-4575-a948-129cf1dd814c None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:43.477003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 665/1342] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:43.482117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0110-abc7-413b-8079-0d3f6dfec9f9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:43.482540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0110-abc7-413b-8079-0d3f6dfec9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:43.482746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0110-abc7-413b-8079-0d3f6dfec9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:43.510889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b6dd0110-abc7-413b-8079-0d3f6dfec9f9 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:43.512025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b6dd0110-abc7-413b-8079-0d3f6dfec9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:43.586670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 678/1343] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:43.624265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48ca4-c9ee-4575-a948-129cf1dd814c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:43.634660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db48ca4-c9ee-4575-a948-129cf1dd814c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:43.638057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03464eac-2b26-43a2-9cd1-197694d0bf60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:43.638683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03464eac-2b26-43a2-9cd1-197694d0bf60 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:43.638954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03464eac-2b26-43a2-9cd1-197694d0bf60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:43.639218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03464eac-2b26-43a2-9cd1-197694d0bf60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:43.647956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03464eac-2b26-43a2-9cd1-197694d0bf60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:43.649370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03464eac-2b26-43a2-9cd1-197694d0bf60 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:43.658427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 666/1344] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:48.079147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e1af7-25b8-455f-ab86-5d49d706cf58 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:48.079710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e1af7-25b8-455f-ab86-5d49d706cf58 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:48.079710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e1af7-25b8-455f-ab86-5d49d706cf58 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:48.081221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 679/1345] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:48.086707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d423-fbea-417d-bb83-109bed1719f7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:48.087116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d423-fbea-417d-bb83-109bed1719f7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:48.087370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d423-fbea-417d-bb83-109bed1719f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:48.105164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6259d423-fbea-417d-bb83-109bed1719f7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:48.106281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6259d423-fbea-417d-bb83-109bed1719f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:48.114521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 667/1346] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:48.151924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e1af7-25b8-455f-ab86-5d49d706cf58 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:06:48.162944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e1af7-25b8-455f-ab86-5d49d706cf58 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:06:48.166418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd431b92-31db-4b42-9ad9-4c39aee99e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:06:48.167049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431b92-31db-4b42-9ad9-4c39aee99e50 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:48.167329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431b92-31db-4b42-9ad9-4c39aee99e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:48.167600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431b92-31db-4b42-9ad9-4c39aee99e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:48.177348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd431b92-31db-4b42-9ad9-4c39aee99e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:06:48.180173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd431b92-31db-4b42-9ad9-4c39aee99e50 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:06:48.188016 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 680/1347] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:06:53.662085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df912826-be82-4aee-b2c6-73d57f49a963 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:53.662606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df912826-be82-4aee-b2c6-73d57f49a963 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:53.662606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df912826-be82-4aee-b2c6-73d57f49a963 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:53.664361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 668/1348] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:53.673399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd49dbb-2e0f-4609-b9b3-48db33a8b09e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:53.673399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd49dbb-2e0f-4609-b9b3-48db33a8b09e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:53.673399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd49dbb-2e0f-4609-b9b3-48db33a8b09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:53.743034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3cd49dbb-2e0f-4609-b9b3-48db33a8b09e None None] MFA Rules not processed for user `6c36906c508f41cca018e5f6dcaadd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:53.744558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3cd49dbb-2e0f-4609-b9b3-48db33a8b09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:53.827194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 681/1349] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:57.992317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e445a-8d35-4e39-a134-40f640b7aec4 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:57.992317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e445a-8d35-4e39-a134-40f640b7aec4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:57.992317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e445a-8d35-4e39-a134-40f640b7aec4 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:57.993494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 669/1350] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:57.998375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0096-d2bf-4adb-aee9-81db75ab43bb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:57.998831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0096-d2bf-4adb-aee9-81db75ab43bb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:57.999120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0096-d2bf-4adb-aee9-81db75ab43bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:58.023396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-297b0096-d2bf-4adb-aee9-81db75ab43bb None None] MFA Rules not processed for user `6c36906c508f41cca018e5f6dcaadd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:58.024768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-297b0096-d2bf-4adb-aee9-81db75ab43bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:58.033605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 682/1351] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:06:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:06:59.680531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe02d0-e0fa-453b-aeee-dac3808fd9cd None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:59.681068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe02d0-e0fa-453b-aeee-dac3808fd9cd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:59.681068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe02d0-e0fa-453b-aeee-dac3808fd9cd None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:59.682410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 670/1352] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:06:59.687250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ceac-ce1b-4d68-8626-71176c0b71e2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:06:59.687698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ceac-ce1b-4d68-8626-71176c0b71e2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:06:59.687825 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ceac-ce1b-4d68-8626-71176c0b71e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:06:59.705361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3070ceac-ce1b-4d68-8626-71176c0b71e2 None None] MFA Rules not processed for user `6c36906c508f41cca018e5f6dcaadd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:06:59.707088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3070ceac-ce1b-4d68-8626-71176c0b71e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:06:59.715399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 683/1353] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:03.666582 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7e0b1-ce52-49a7-a047-27717073f01b None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:03.667110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7e0b1-ce52-49a7-a047-27717073f01b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:03.667110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7e0b1-ce52-49a7-a047-27717073f01b None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:03.672529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 671/1354] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:03.674251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74e290-052c-4fa9-9007-e2cddf5ad7da None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:03.675604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74e290-052c-4fa9-9007-e2cddf5ad7da None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:03.676077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74e290-052c-4fa9-9007-e2cddf5ad7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:03.697955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ba74e290-052c-4fa9-9007-e2cddf5ad7da None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:03.699253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ba74e290-052c-4fa9-9007-e2cddf5ad7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:03.708034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 684/1355] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:07:03.756517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db34ca-147e-4489-b695-f1dbf5cdc18f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:03.756923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db34ca-147e-4489-b695-f1dbf5cdc18f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:03.758294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db34ca-147e-4489-b695-f1dbf5cdc18f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:03.759951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 672/1356] 10.222.0.71 () {54 vars in 961 bytes} [Mon May 16 15:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:03.765783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12597-3888-44fa-8397-6d8d4c950831 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:03.766074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12597-3888-44fa-8397-6d8d4c950831 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:03.766351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12597-3888-44fa-8397-6d8d4c950831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:03.825124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c1e12597-3888-44fa-8397-6d8d4c950831 None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:03.826295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c1e12597-3888-44fa-8397-6d8d4c950831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:03.912345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 685/1357] 10.222.0.71 () {58 vars in 1058 bytes} [Mon May 16 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:03.920111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db34ca-147e-4489-b695-f1dbf5cdc18f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:03.936249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db34ca-147e-4489-b695-f1dbf5cdc18f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:03.942311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186d76-4dfe-4ef3-afa4-2cd9459e22ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:03.942953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186d76-4dfe-4ef3-afa4-2cd9459e22ed service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:03.943231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186d76-4dfe-4ef3-afa4-2cd9459e22ed service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:03.943500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186d76-4dfe-4ef3-afa4-2cd9459e22ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:03.951921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186d76-4dfe-4ef3-afa4-2cd9459e22ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:03.953488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186d76-4dfe-4ef3-afa4-2cd9459e22ed service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:03.963100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 673/1358] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:06.415943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed9c87-c8df-4e69-a441-a32e76f7a4ef None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:06.416507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed9c87-c8df-4e69-a441-a32e76f7a4ef None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:06.416507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed9c87-c8df-4e69-a441-a32e76f7a4ef None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:06.417937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 686/1359] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:06.423254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7f83-9a97-4af1-a169-4ed090425e14 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:06.423711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7f83-9a97-4af1-a169-4ed090425e14 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:06.423834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7f83-9a97-4af1-a169-4ed090425e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:06.442534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d08b7f83-9a97-4af1-a169-4ed090425e14 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:06.443993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d08b7f83-9a97-4af1-a169-4ed090425e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:06.452725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 674/1360] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:06.496542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed9c87-c8df-4e69-a441-a32e76f7a4ef None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:06.516102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed9c87-c8df-4e69-a441-a32e76f7a4ef None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:06.525518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d695578-1f83-4f28-82de-599bc25af043 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:06.525518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d695578-1f83-4f28-82de-599bc25af043 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:06.525518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d695578-1f83-4f28-82de-599bc25af043 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:06.525518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d695578-1f83-4f28-82de-599bc25af043 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:06.532748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d695578-1f83-4f28-82de-599bc25af043 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:06.534535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d695578-1f83-4f28-82de-599bc25af043 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:06.544250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 687/1361] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:09.192168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f93bac-9e67-4130-90f6-01142573763b None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:09.192669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f93bac-9e67-4130-90f6-01142573763b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:09.192669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f93bac-9e67-4130-90f6-01142573763b None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:09.194259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 675/1362] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:09.199745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b8671-b5df-42a9-b208-bb7a955b2710 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:09.200186 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b8671-b5df-42a9-b208-bb7a955b2710 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:09.200279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b8671-b5df-42a9-b208-bb7a955b2710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:09.218016 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-801b8671-b5df-42a9-b208-bb7a955b2710 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:09.219211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-801b8671-b5df-42a9-b208-bb7a955b2710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:09.227913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 688/1363] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:09.273018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdbed1-6ef4-4b97-93a0-58e8088f3607 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:09.273305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdbed1-6ef4-4b97-93a0-58e8088f3607 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:09.273580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdbed1-6ef4-4b97-93a0-58e8088f3607 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:09.274962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 676/1364] 10.222.0.71 () {54 vars in 961 bytes} [Mon May 16 15:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:09.280359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d6687-4353-4132-aa01-08e61f884caf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:09.280648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d6687-4353-4132-aa01-08e61f884caf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:09.280922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d6687-4353-4132-aa01-08e61f884caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:09.299828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fc1d6687-4353-4132-aa01-08e61f884caf None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:09.300993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc1d6687-4353-4132-aa01-08e61f884caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:09.309918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 689/1365] 10.222.0.71 () {58 vars in 1058 bytes} [Mon May 16 15:07:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:09.317596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdbed1-6ef4-4b97-93a0-58e8088f3607 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:09.331590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdbed1-6ef4-4b97-93a0-58e8088f3607 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:09.336977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40579ab4-1b7e-4dc3-b365-320d4dd0749e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:09.337598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40579ab4-1b7e-4dc3-b365-320d4dd0749e service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:09.337874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40579ab4-1b7e-4dc3-b365-320d4dd0749e service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:09.338143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40579ab4-1b7e-4dc3-b365-320d4dd0749e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:09.347021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40579ab4-1b7e-4dc3-b365-320d4dd0749e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:09.348710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40579ab4-1b7e-4dc3-b365-320d4dd0749e service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:09.358133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 677/1366] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:11.805164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b188-0c99-4d03-8a4e-bdaaffb28fc8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:11.805698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b188-0c99-4d03-8a4e-bdaaffb28fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:11.805698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b188-0c99-4d03-8a4e-bdaaffb28fc8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:11.807072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 690/1367] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:11.812456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0612751-3e92-4dab-b9fb-d4d818b74c45 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:11.812870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0612751-3e92-4dab-b9fb-d4d818b74c45 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:11.812998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0612751-3e92-4dab-b9fb-d4d818b74c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:11.832337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f0612751-3e92-4dab-b9fb-d4d818b74c45 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:11.833708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f0612751-3e92-4dab-b9fb-d4d818b74c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:11.848301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 678/1368] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:11.886354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bf4a0-b2d2-439e-a152-1c7636788c7d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:11.887006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bf4a0-b2d2-439e-a152-1c7636788c7d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:11.887374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bf4a0-b2d2-439e-a152-1c7636788c7d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:11.888948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 691/1369] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:11.890528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0612751-3e92-4dab-b9fb-d4d818b74c45 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:11.894294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0d55-49c1-46ff-9f95-be97c6eeeff4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:11.894580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0d55-49c1-46ff-9f95-be97c6eeeff4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:11.894852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0d55-49c1-46ff-9f95-be97c6eeeff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:11.907368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0612751-3e92-4dab-b9fb-d4d818b74c45 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:11.912933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-95dd0d55-49c1-46ff-9f95-be97c6eeeff4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:11.913128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0a677-a2b6-49fc-a62b-6a3de3374d46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:11.913881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0a677-a2b6-49fc-a62b-6a3de3374d46 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:11.914138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95dd0d55-49c1-46ff-9f95-be97c6eeeff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:11.914340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0a677-a2b6-49fc-a62b-6a3de3374d46 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:11.914725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0a677-a2b6-49fc-a62b-6a3de3374d46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:11.923157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 692/1370] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:11.929069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0a677-a2b6-49fc-a62b-6a3de3374d46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:11.930875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0a677-a2b6-49fc-a62b-6a3de3374d46 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:11.947814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 679/1371] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:12.241690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afedaf0-938c-4312-a45a-3c213268a850 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:12.241690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afedaf0-938c-4312-a45a-3c213268a850 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:12.241690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afedaf0-938c-4312-a45a-3c213268a850 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:12.241690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 693/1372] 10.222.0.71 () {54 vars in 963 bytes} [Mon May 16 15:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:12.247020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb346df-e35a-48b3-a1e6-cb5a51366dbe None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:12.247388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb346df-e35a-48b3-a1e6-cb5a51366dbe None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:12.247592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb346df-e35a-48b3-a1e6-cb5a51366dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:12.302641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ceb346df-e35a-48b3-a1e6-cb5a51366dbe None None] MFA Rules not processed for user `1f2421c1bfb5402da3179e6f3c749374`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:12.303345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ceb346df-e35a-48b3-a1e6-cb5a51366dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:12.363421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 680/1373] 10.222.0.71 () {58 vars in 1060 bytes} [Mon May 16 15:07:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:12.370769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afedaf0-938c-4312-a45a-3c213268a850 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:12.381913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afedaf0-938c-4312-a45a-3c213268a850 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:12.385800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbeffb-c001-4001-a62b-457a41a6c5e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:12.386542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbeffb-c001-4001-a62b-457a41a6c5e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:12.386891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbeffb-c001-4001-a62b-457a41a6c5e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:12.387236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbeffb-c001-4001-a62b-457a41a6c5e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:12.397278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbeffb-c001-4001-a62b-457a41a6c5e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:12.398791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbeffb-c001-4001-a62b-457a41a6c5e0 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:12.408899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 694/1374] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:07:14.356768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169b51-026f-486f-a5a4-33a1d039e774 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:14.357313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169b51-026f-486f-a5a4-33a1d039e774 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:14.357313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169b51-026f-486f-a5a4-33a1d039e774 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:14.358710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 681/1375] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:14.364866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e642f52-6d44-4e0c-8a71-c55a112fdc17 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:14.365293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e642f52-6d44-4e0c-8a71-c55a112fdc17 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:14.365411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e642f52-6d44-4e0c-8a71-c55a112fdc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:14.383743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7e642f52-6d44-4e0c-8a71-c55a112fdc17 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:14.384947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7e642f52-6d44-4e0c-8a71-c55a112fdc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:14.392865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 695/1376] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:14.437965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169b51-026f-486f-a5a4-33a1d039e774 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:14.452875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169b51-026f-486f-a5a4-33a1d039e774 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:14.458795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44af821-0c93-4404-a4c9-48c78294c0c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:14.459513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44af821-0c93-4404-a4c9-48c78294c0c3 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:14.459863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44af821-0c93-4404-a4c9-48c78294c0c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:14.460165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44af821-0c93-4404-a4c9-48c78294c0c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:14.468546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44af821-0c93-4404-a4c9-48c78294c0c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:14.470793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44af821-0c93-4404-a4c9-48c78294c0c3 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:14.480499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 682/1377] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:16.931981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcef8c-d5aa-4cbf-95f4-263932ce76d7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:16.932537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcef8c-d5aa-4cbf-95f4-263932ce76d7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:16.932795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcef8c-d5aa-4cbf-95f4-263932ce76d7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:16.934283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 696/1378] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:16.940195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e129a-ab9a-4c3a-8708-53649b058466 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:16.940599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e129a-ab9a-4c3a-8708-53649b058466 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:16.940747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e129a-ab9a-4c3a-8708-53649b058466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:16.970724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-215e129a-ab9a-4c3a-8708-53649b058466 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:16.978918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-215e129a-ab9a-4c3a-8708-53649b058466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:16.980030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 683/1379] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:17.023784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcef8c-d5aa-4cbf-95f4-263932ce76d7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:17.038269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcef8c-d5aa-4cbf-95f4-263932ce76d7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:17.043730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635195-2b5b-4b7b-ba2d-20f6d54ccb85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:17.044373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635195-2b5b-4b7b-ba2d-20f6d54ccb85 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:17.044659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635195-2b5b-4b7b-ba2d-20f6d54ccb85 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:17.045202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635195-2b5b-4b7b-ba2d-20f6d54ccb85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:17.054148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635195-2b5b-4b7b-ba2d-20f6d54ccb85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:17.055762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635195-2b5b-4b7b-ba2d-20f6d54ccb85 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:17.067045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 697/1380] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:19.459076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058733a-7529-4155-a20f-a4159dae08ca None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:19.459652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058733a-7529-4155-a20f-a4159dae08ca None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:19.459652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058733a-7529-4155-a20f-a4159dae08ca None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:19.461076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 684/1381] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:19.466462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980d5f4-fe10-4af4-a8fa-0d11a3457de4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:19.466892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980d5f4-fe10-4af4-a8fa-0d11a3457de4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:19.467063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980d5f4-fe10-4af4-a8fa-0d11a3457de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:19.484366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7980d5f4-fe10-4af4-a8fa-0d11a3457de4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:19.486198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7980d5f4-fe10-4af4-a8fa-0d11a3457de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:19.494669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 698/1382] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:19.536573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058733a-7529-4155-a20f-a4159dae08ca None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:19.552789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058733a-7529-4155-a20f-a4159dae08ca None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:19.558579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2f3f5-8aaa-4190-a68f-238467ee7158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:19.559359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2f3f5-8aaa-4190-a68f-238467ee7158 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:19.559770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2f3f5-8aaa-4190-a68f-238467ee7158 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:19.560148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2f3f5-8aaa-4190-a68f-238467ee7158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:19.568776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2f3f5-8aaa-4190-a68f-238467ee7158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:19.570605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2f3f5-8aaa-4190-a68f-238467ee7158 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:19.580442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 685/1383] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:21.823793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54339e8-15ee-4ec3-9cdd-ed255761d1b8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:21.824349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54339e8-15ee-4ec3-9cdd-ed255761d1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:21.824445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54339e8-15ee-4ec3-9cdd-ed255761d1b8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:21.825790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 699/1384] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:21.832391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107e85d-b75f-4c5a-8b65-4228419b0d36 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:21.832842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107e85d-b75f-4c5a-8b65-4228419b0d36 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:21.833125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107e85d-b75f-4c5a-8b65-4228419b0d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:21.851605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f107e85d-b75f-4c5a-8b65-4228419b0d36 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:21.852819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f107e85d-b75f-4c5a-8b65-4228419b0d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:21.862620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 686/1385] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:21.909821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54339e8-15ee-4ec3-9cdd-ed255761d1b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:21.924627 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54339e8-15ee-4ec3-9cdd-ed255761d1b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:21.930157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f655c-d289-4e07-a70f-4d1519884ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:21.930798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f655c-d289-4e07-a70f-4d1519884ebf service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:21.931082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f655c-d289-4e07-a70f-4d1519884ebf service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:21.931359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f655c-d289-4e07-a70f-4d1519884ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:21.940252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f655c-d289-4e07-a70f-4d1519884ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:21.941847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f655c-d289-4e07-a70f-4d1519884ebf service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:21.951296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 700/1386] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:24.510439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6dbfe-1fea-461b-a57a-b800f0e5d5b0 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:24.511008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6dbfe-1fea-461b-a57a-b800f0e5d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:24.511008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6dbfe-1fea-461b-a57a-b800f0e5d5b0 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:24.512383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 687/1387] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:24.518454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aea2fd-6311-494f-a511-30cad92c0dc8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:24.518864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aea2fd-6311-494f-a511-30cad92c0dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:24.519013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aea2fd-6311-494f-a511-30cad92c0dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:24.537343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-65aea2fd-6311-494f-a511-30cad92c0dc8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:24.538542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-65aea2fd-6311-494f-a511-30cad92c0dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:24.547182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 701/1388] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:24.591035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6dbfe-1fea-461b-a57a-b800f0e5d5b0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:24.605788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6dbfe-1fea-461b-a57a-b800f0e5d5b0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:24.611740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87500d21-a855-47b1-a1df-b66bac9a885a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:24.612379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87500d21-a855-47b1-a1df-b66bac9a885a service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:24.612656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87500d21-a855-47b1-a1df-b66bac9a885a service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:24.612927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87500d21-a855-47b1-a1df-b66bac9a885a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:24.622152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87500d21-a855-47b1-a1df-b66bac9a885a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:24.624303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87500d21-a855-47b1-a1df-b66bac9a885a service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:24.636863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 688/1389] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:27.221959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3c473-2633-4d10-a48d-dd9746fbe8a4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:27.222552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3c473-2633-4d10-a48d-dd9746fbe8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:27.222552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3c473-2633-4d10-a48d-dd9746fbe8a4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:27.223931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 702/1390] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:27.229238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3649b3-e225-4d9c-8cf9-a3049a0f3717 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:27.229688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3649b3-e225-4d9c-8cf9-a3049a0f3717 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:27.229863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3649b3-e225-4d9c-8cf9-a3049a0f3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:27.247735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1a3649b3-e225-4d9c-8cf9-a3049a0f3717 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:27.248919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1a3649b3-e225-4d9c-8cf9-a3049a0f3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:27.256789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 689/1391] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:27.301020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3c473-2633-4d10-a48d-dd9746fbe8a4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:27.315193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3c473-2633-4d10-a48d-dd9746fbe8a4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:27.320647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59ec86-6c80-4f20-81ed-ef24b8ef597d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:27.321285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59ec86-6c80-4f20-81ed-ef24b8ef597d service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:27.321572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59ec86-6c80-4f20-81ed-ef24b8ef597d service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:27.321850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59ec86-6c80-4f20-81ed-ef24b8ef597d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:27.330884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59ec86-6c80-4f20-81ed-ef24b8ef597d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:27.332579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59ec86-6c80-4f20-81ed-ef24b8ef597d service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:27.342980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 703/1392] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:28.440273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376680e4-e41d-4b58-8aa4-cce6feb35047 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:28.440744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376680e4-e41d-4b58-8aa4-cce6feb35047 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:28.440849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376680e4-e41d-4b58-8aa4-cce6feb35047 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:28.442201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 690/1393] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:28.447442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c511d3-fab7-422b-8a13-3f67a05da96c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:28.447881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c511d3-fab7-422b-8a13-3f67a05da96c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:28.448043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c511d3-fab7-422b-8a13-3f67a05da96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:28.465945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-46c511d3-fab7-422b-8a13-3f67a05da96c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:28.467155 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-46c511d3-fab7-422b-8a13-3f67a05da96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:28.475857 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 704/1394] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:28.530410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322c2d-60bb-442f-bdcf-b4bc30b3e8f2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:28.530695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322c2d-60bb-442f-bdcf-b4bc30b3e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:28.531617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322c2d-60bb-442f-bdcf-b4bc30b3e8f2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:28.532998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 691/1395] 10.222.0.71 () {54 vars in 962 bytes} [Mon May 16 15:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:28.538238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8405-a7f2-41a4-bf38-dc1070b0ea30 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:28.538597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8405-a7f2-41a4-bf38-dc1070b0ea30 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:28.539001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8405-a7f2-41a4-bf38-dc1070b0ea30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:28.586003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-410a8405-a7f2-41a4-bf38-dc1070b0ea30 None None] MFA Rules not processed for user `b60d4a75e5b8464a8c7935e7433cf170`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:28.587181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-410a8405-a7f2-41a4-bf38-dc1070b0ea30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:28.644779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 705/1396] 10.222.0.71 () {58 vars in 1059 bytes} [Mon May 16 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:28.652148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e322c2d-60bb-442f-bdcf-b4bc30b3e8f2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:28.662741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e322c2d-60bb-442f-bdcf-b4bc30b3e8f2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:28.666412 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb261d-88c5-44bf-96a7-ccef5f10f962 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:28.667043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb261d-88c5-44bf-96a7-ccef5f10f962 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:28.667320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb261d-88c5-44bf-96a7-ccef5f10f962 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:28.667594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb261d-88c5-44bf-96a7-ccef5f10f962 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:28.677368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb261d-88c5-44bf-96a7-ccef5f10f962 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:28.678754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb261d-88c5-44bf-96a7-ccef5f10f962 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:28.689919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 692/1397] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:29.260190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15914a-acc0-4e30-ae7e-8324bfb5c5a6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:29.260481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15914a-acc0-4e30-ae7e-8324bfb5c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:29.260761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15914a-acc0-4e30-ae7e-8324bfb5c5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:29.309385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4c15914a-acc0-4e30-ae7e-8324bfb5c5a6 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:29.310600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c15914a-acc0-4e30-ae7e-8324bfb5c5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:29.373495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 706/1398] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:07:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:29.418426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c4895-c4df-445f-940b-d569b04c5528 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:29.422955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c4895-c4df-445f-940b-d569b04c5528 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:29.422955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c4895-c4df-445f-940b-d569b04c5528 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:29.422955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 693/1399] 10.222.0.71 () {54 vars in 942 bytes} [Mon May 16 15:07:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:29.428899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea665-7fb4-4dfb-9a4b-9a0899db79b7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:29.429442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea665-7fb4-4dfb-9a4b-9a0899db79b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:29.429906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea665-7fb4-4dfb-9a4b-9a0899db79b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:29.478666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f06ea665-7fb4-4dfb-9a4b-9a0899db79b7 None None] MFA Rules not processed for user `918d67038f9c4d0fa4d2456409245774`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:29.479944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f06ea665-7fb4-4dfb-9a4b-9a0899db79b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:29.550509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 707/1400] 10.222.0.71 () {58 vars in 1039 bytes} [Mon May 16 15:07:29 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:29.559850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c4895-c4df-445f-940b-d569b04c5528 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:29.572913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c4895-c4df-445f-940b-d569b04c5528 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:29.577659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1b470-ccb5-4897-a54c-ba05433bd92e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:29.578323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1b470-ccb5-4897-a54c-ba05433bd92e service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:29.578608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1b470-ccb5-4897-a54c-ba05433bd92e service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:29.578866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1b470-ccb5-4897-a54c-ba05433bd92e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:29.586707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1b470-ccb5-4897-a54c-ba05433bd92e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:29.588311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1b470-ccb5-4897-a54c-ba05433bd92e service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:29.596627 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 694/1401] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:07:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:07:29.813463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600efbb-8b13-4f00-b40c-b9216599da6e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:29.813675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600efbb-8b13-4f00-b40c-b9216599da6e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:29.813927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600efbb-8b13-4f00-b40c-b9216599da6e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:29.815329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 708/1402] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:29.820444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3683004-0c13-445f-8300-96fea5b7f8e0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:29.820734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3683004-0c13-445f-8300-96fea5b7f8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:29.821011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3683004-0c13-445f-8300-96fea5b7f8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:29.839364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e3683004-0c13-445f-8300-96fea5b7f8e0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:29.840606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e3683004-0c13-445f-8300-96fea5b7f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:29.849251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 695/1403] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:29.892580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600efbb-8b13-4f00-b40c-b9216599da6e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:29.920243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600efbb-8b13-4f00-b40c-b9216599da6e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:29.926094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc86a8-fdac-47e4-98b1-a03b9250ba76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:29.926874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc86a8-fdac-47e4-98b1-a03b9250ba76 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:29.927160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc86a8-fdac-47e4-98b1-a03b9250ba76 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:29.927440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc86a8-fdac-47e4-98b1-a03b9250ba76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:29.937887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc86a8-fdac-47e4-98b1-a03b9250ba76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:29.939493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc86a8-fdac-47e4-98b1-a03b9250ba76 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:29.953650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 709/1404] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:32.519357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171141-3c7e-4f57-99e3-0a6f4c7bd2c4 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:32.519990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171141-3c7e-4f57-99e3-0a6f4c7bd2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:32.519990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171141-3c7e-4f57-99e3-0a6f4c7bd2c4 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:32.521926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 696/1405] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:32.529493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fa5ac-3065-4d37-85ae-88b2cf8975b1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:32.529955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fa5ac-3065-4d37-85ae-88b2cf8975b1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:32.530058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fa5ac-3065-4d37-85ae-88b2cf8975b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:32.551466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0c0fa5ac-3065-4d37-85ae-88b2cf8975b1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:32.551466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0c0fa5ac-3065-4d37-85ae-88b2cf8975b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:32.559672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 710/1406] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:32.605763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e171141-3c7e-4f57-99e3-0a6f4c7bd2c4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:32.625113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e171141-3c7e-4f57-99e3-0a6f4c7bd2c4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:32.631457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041f0b1-5066-48b6-a224-fdf1ed731b01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:32.632469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041f0b1-5066-48b6-a224-fdf1ed731b01 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:32.632972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041f0b1-5066-48b6-a224-fdf1ed731b01 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:32.633435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041f0b1-5066-48b6-a224-fdf1ed731b01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:32.644980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041f0b1-5066-48b6-a224-fdf1ed731b01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:32.647310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041f0b1-5066-48b6-a224-fdf1ed731b01 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:32.659185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 697/1407] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:53.923221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76d808-1fc1-49d5-85cb-a5687075e441 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:53.923985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76d808-1fc1-49d5-85cb-a5687075e441 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:53.923985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76d808-1fc1-49d5-85cb-a5687075e441 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:53.925281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 711/1408] 10.222.0.71 () {54 vars in 918 bytes} [Mon May 16 15:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:53.930394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f8184-50e7-4cde-b549-3eb143123c77 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:53.930894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f8184-50e7-4cde-b549-3eb143123c77 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:53.931000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f8184-50e7-4cde-b549-3eb143123c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:53.948942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-607f8184-50e7-4cde-b549-3eb143123c77 None None] MFA Rules not processed for user `6c36906c508f41cca018e5f6dcaadd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:53.950132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-607f8184-50e7-4cde-b549-3eb143123c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:53.958704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 698/1409] 10.222.0.71 () {58 vars in 1015 bytes} [Mon May 16 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:53.973158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0804-4780-4137-8e29-b1dbd87f2740 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:53.973466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0804-4780-4137-8e29-b1dbd87f2740 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:53.973747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0804-4780-4137-8e29-b1dbd87f2740 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:53.975112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 712/1410] 10.222.0.71 () {54 vars in 918 bytes} [Mon May 16 15:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:53.979551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0404d1b-981f-4d72-b5e9-3ef25188346c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:53.982787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0404d1b-981f-4d72-b5e9-3ef25188346c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:53.982787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0404d1b-981f-4d72-b5e9-3ef25188346c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:53.997640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a0404d1b-981f-4d72-b5e9-3ef25188346c None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:53.998798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a0404d1b-981f-4d72-b5e9-3ef25188346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:54.007964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 699/1411] 10.222.0.71 () {58 vars in 1015 bytes} [Mon May 16 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:56.697485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767584b8-fd51-4e0d-a0f4-23080ca0fba1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:56.698059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767584b8-fd51-4e0d-a0f4-23080ca0fba1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:56.698059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767584b8-fd51-4e0d-a0f4-23080ca0fba1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:56.699398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 713/1412] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:56.705112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e81ec-25ed-4083-a4a3-ec786c6ed81d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:56.705563 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e81ec-25ed-4083-a4a3-ec786c6ed81d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:56.705669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e81ec-25ed-4083-a4a3-ec786c6ed81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:56.722554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-504e81ec-25ed-4083-a4a3-ec786c6ed81d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:56.723781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-504e81ec-25ed-4083-a4a3-ec786c6ed81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:56.731372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 700/1413] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:56.773573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767584b8-fd51-4e0d-a0f4-23080ca0fba1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:56.787367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767584b8-fd51-4e0d-a0f4-23080ca0fba1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:56.792959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fef31c-5c41-437a-886e-671ed301814d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:56.793583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fef31c-5c41-437a-886e-671ed301814d service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:56.793860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fef31c-5c41-437a-886e-671ed301814d service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:56.794130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fef31c-5c41-437a-886e-671ed301814d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:56.802422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fef31c-5c41-437a-886e-671ed301814d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:56.804025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fef31c-5c41-437a-886e-671ed301814d service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:56.813638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 714/1414] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:57.808300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9034e-eb10-49d7-915d-2c870fb3a91c None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:57.808871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9034e-eb10-49d7-915d-2c870fb3a91c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:57.808871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9034e-eb10-49d7-915d-2c870fb3a91c None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:57.810196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 701/1415] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:57.815821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5417b3-53ed-45e8-8a7b-2124264b46a0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:57.816212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5417b3-53ed-45e8-8a7b-2124264b46a0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:57.816375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5417b3-53ed-45e8-8a7b-2124264b46a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:57.834462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7f5417b3-53ed-45e8-8a7b-2124264b46a0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:57.835663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7f5417b3-53ed-45e8-8a7b-2124264b46a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:57.843742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 715/1416] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:58.002274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9034e-eb10-49d7-915d-2c870fb3a91c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:58.016671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9034e-eb10-49d7-915d-2c870fb3a91c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:58.025865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912f835-079b-40b4-a7b4-30870744e871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:58.025865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912f835-079b-40b4-a7b4-30870744e871 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:58.025865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912f835-079b-40b4-a7b4-30870744e871 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:58.025865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912f835-079b-40b4-a7b4-30870744e871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:58.030634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912f835-079b-40b4-a7b4-30870744e871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:58.032226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912f835-079b-40b4-a7b4-30870744e871 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:58.041445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 702/1417] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:07:59.005154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4777e-12a6-4cd5-ab51-c8e56e361616 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:59.005734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4777e-12a6-4cd5-ab51-c8e56e361616 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:59.005734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4777e-12a6-4cd5-ab51-c8e56e361616 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:59.007172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 716/1418] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:07:59.012364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a131e-56e5-4d3f-b705-a9f62be92a0f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:59.012793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a131e-56e5-4d3f-b705-a9f62be92a0f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:59.012891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a131e-56e5-4d3f-b705-a9f62be92a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:59.031530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-cd7a131e-56e5-4d3f-b705-a9f62be92a0f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:07:59.032790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cd7a131e-56e5-4d3f-b705-a9f62be92a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:07:59.040735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 703/1419] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:07:59.189923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4777e-12a6-4cd5-ab51-c8e56e361616 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:07:59.203734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4777e-12a6-4cd5-ab51-c8e56e361616 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:07:59.208976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905868e7-31f6-4b9b-b5ec-dfdd73102945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:07:59.209601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905868e7-31f6-4b9b-b5ec-dfdd73102945 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:07:59.209878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905868e7-31f6-4b9b-b5ec-dfdd73102945 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:07:59.210149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905868e7-31f6-4b9b-b5ec-dfdd73102945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:07:59.218423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905868e7-31f6-4b9b-b5ec-dfdd73102945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:07:59.220031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905868e7-31f6-4b9b-b5ec-dfdd73102945 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:07:59.229031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 717/1420] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:07:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:00.148154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341750-abea-4535-b6d9-dfa2d35ac848 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:00.148665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341750-abea-4535-b6d9-dfa2d35ac848 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:00.148665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341750-abea-4535-b6d9-dfa2d35ac848 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:00.149980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 704/1421] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:00.154822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030d63e-c4ae-407c-b26f-424eed7db9e6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:00.155738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030d63e-c4ae-407c-b26f-424eed7db9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:00.155896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030d63e-c4ae-407c-b26f-424eed7db9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:00.174674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0030d63e-c4ae-407c-b26f-424eed7db9e6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:00.175884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0030d63e-c4ae-407c-b26f-424eed7db9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:00.183700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 718/1422] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:00.334229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341750-abea-4535-b6d9-dfa2d35ac848 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:00.348698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341750-abea-4535-b6d9-dfa2d35ac848 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:00.353898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472520f-3d56-42da-987e-585f1598d7a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:00.354520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472520f-3d56-42da-987e-585f1598d7a2 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:00.354796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472520f-3d56-42da-987e-585f1598d7a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:00.355067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472520f-3d56-42da-987e-585f1598d7a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:00.362792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472520f-3d56-42da-987e-585f1598d7a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:00.364382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472520f-3d56-42da-987e-585f1598d7a2 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:00.373460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 705/1423] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:01.308096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23989ed0-1168-4522-926a-cb25371b0313 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:01.308518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23989ed0-1168-4522-926a-cb25371b0313 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:01.308648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23989ed0-1168-4522-926a-cb25371b0313 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:01.310021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 719/1424] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:01.315104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bebbe-e4b4-4ba5-a77a-11970e33e5e6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:01.315511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bebbe-e4b4-4ba5-a77a-11970e33e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:01.315684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bebbe-e4b4-4ba5-a77a-11970e33e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:01.333581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9f1bebbe-e4b4-4ba5-a77a-11970e33e5e6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:01.334759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9f1bebbe-e4b4-4ba5-a77a-11970e33e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:01.342953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 706/1425] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:01.491496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23989ed0-1168-4522-926a-cb25371b0313 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:01.505919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23989ed0-1168-4522-926a-cb25371b0313 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:01.511112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a91552-4a8d-4eeb-96b6-3a053bbf561f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:01.511771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a91552-4a8d-4eeb-96b6-3a053bbf561f service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:01.512053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a91552-4a8d-4eeb-96b6-3a053bbf561f service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:01.512325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a91552-4a8d-4eeb-96b6-3a053bbf561f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:01.520872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a91552-4a8d-4eeb-96b6-3a053bbf561f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:01.522427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a91552-4a8d-4eeb-96b6-3a053bbf561f service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:01.531485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 720/1426] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:02.497483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b15bd-4b9f-47b9-991e-b03bd8582f31 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:02.498054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b15bd-4b9f-47b9-991e-b03bd8582f31 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:02.498054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b15bd-4b9f-47b9-991e-b03bd8582f31 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:02.499902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 707/1427] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:02.504779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb308e0-6336-40c9-a253-1ccb45d58165 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:02.505241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb308e0-6336-40c9-a253-1ccb45d58165 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:02.505350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb308e0-6336-40c9-a253-1ccb45d58165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:02.523258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7fb308e0-6336-40c9-a253-1ccb45d58165 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:02.524457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7fb308e0-6336-40c9-a253-1ccb45d58165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:02.532314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 721/1428] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:02.681513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b15bd-4b9f-47b9-991e-b03bd8582f31 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:02.695961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b15bd-4b9f-47b9-991e-b03bd8582f31 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:02.701236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227bd56-13ab-4593-a888-4e68b22f2c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:02.701887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227bd56-13ab-4593-a888-4e68b22f2c7e service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:02.702162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227bd56-13ab-4593-a888-4e68b22f2c7e service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:02.702429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227bd56-13ab-4593-a888-4e68b22f2c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:02.710181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227bd56-13ab-4593-a888-4e68b22f2c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:02.711760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227bd56-13ab-4593-a888-4e68b22f2c7e service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:02.720744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 708/1429] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:03.677329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644e03-441a-4872-a0c1-92af1a559894 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:03.677855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644e03-441a-4872-a0c1-92af1a559894 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:03.677855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644e03-441a-4872-a0c1-92af1a559894 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:03.679171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 722/1430] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:03.684254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ee83-d4d5-4477-ade0-377e0b6edaed None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:03.684666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ee83-d4d5-4477-ade0-377e0b6edaed None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:03.684800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ee83-d4d5-4477-ade0-377e0b6edaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:03.703580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0683ee83-d4d5-4477-ade0-377e0b6edaed None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:03.704774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0683ee83-d4d5-4477-ade0-377e0b6edaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:03.712309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 709/1431] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:03.862564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644e03-441a-4872-a0c1-92af1a559894 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:03.877135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644e03-441a-4872-a0c1-92af1a559894 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:03.882401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b9a7e-9f6e-41d8-8efd-139e4f786d0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:03.883036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b9a7e-9f6e-41d8-8efd-139e4f786d0e service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:03.883318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b9a7e-9f6e-41d8-8efd-139e4f786d0e service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:03.883591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b9a7e-9f6e-41d8-8efd-139e4f786d0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:03.892151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b9a7e-9f6e-41d8-8efd-139e4f786d0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:03.893720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b9a7e-9f6e-41d8-8efd-139e4f786d0e service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:03.902729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 723/1432] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:04.898424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab7869-8a4d-4556-a889-1a9fd3485504 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:04.898925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab7869-8a4d-4556-a889-1a9fd3485504 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:04.899023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab7869-8a4d-4556-a889-1a9fd3485504 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:04.900421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 710/1433] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:04.906160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e43e79-b376-4de5-a7db-206abed2c9d8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:04.906575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e43e79-b376-4de5-a7db-206abed2c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:04.906740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e43e79-b376-4de5-a7db-206abed2c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:04.924445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b1e43e79-b376-4de5-a7db-206abed2c9d8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:04.925623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b1e43e79-b376-4de5-a7db-206abed2c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:04.933227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 724/1434] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:05.083567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab7869-8a4d-4556-a889-1a9fd3485504 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:05.098002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab7869-8a4d-4556-a889-1a9fd3485504 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:05.103142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dda36d-8928-4926-ae18-a4a8f05b9df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:05.103814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dda36d-8928-4926-ae18-a4a8f05b9df0 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:05.104094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dda36d-8928-4926-ae18-a4a8f05b9df0 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:05.104366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dda36d-8928-4926-ae18-a4a8f05b9df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:05.112282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dda36d-8928-4926-ae18-a4a8f05b9df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:05.113849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dda36d-8928-4926-ae18-a4a8f05b9df0 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:05.122827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 711/1435] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:06.089039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5633e-7a8d-489f-86b7-91ad5aa3b168 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:06.089573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5633e-7a8d-489f-86b7-91ad5aa3b168 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:06.089573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5633e-7a8d-489f-86b7-91ad5aa3b168 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:06.090947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 725/1436] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:06.096694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dee50-9d49-4034-bb73-ada105d85135 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:06.097128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dee50-9d49-4034-bb73-ada105d85135 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:06.097234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dee50-9d49-4034-bb73-ada105d85135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:06.115522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-962dee50-9d49-4034-bb73-ada105d85135 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:06.116729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-962dee50-9d49-4034-bb73-ada105d85135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:06.124276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 712/1437] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:06.270768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5633e-7a8d-489f-86b7-91ad5aa3b168 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:06.285240 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5633e-7a8d-489f-86b7-91ad5aa3b168 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:06.290646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1c757-c77c-48d0-8aea-fde87fb65613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:06.291272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1c757-c77c-48d0-8aea-fde87fb65613 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:06.291552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1c757-c77c-48d0-8aea-fde87fb65613 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:06.291853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1c757-c77c-48d0-8aea-fde87fb65613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:06.300266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1c757-c77c-48d0-8aea-fde87fb65613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:06.301833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1c757-c77c-48d0-8aea-fde87fb65613 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:06.311113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 726/1438] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:08.395813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913723a-cc37-42e0-a0e3-5bae0ee8e063 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:08.396383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913723a-cc37-42e0-a0e3-5bae0ee8e063 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:08.396383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913723a-cc37-42e0-a0e3-5bae0ee8e063 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:08.397774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 713/1439] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:08.403192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2f84-1e8b-4472-a3fe-fc61a80d03d5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:08.403614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2f84-1e8b-4472-a3fe-fc61a80d03d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:08.403790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2f84-1e8b-4472-a3fe-fc61a80d03d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:08.421261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4bcf2f84-1e8b-4472-a3fe-fc61a80d03d5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:08.422432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4bcf2f84-1e8b-4472-a3fe-fc61a80d03d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:08.430039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 727/1440] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:08.483893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de745b-8d96-4e0c-ae35-5b928760a6d8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:08.484175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de745b-8d96-4e0c-ae35-5b928760a6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:08.484473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de745b-8d96-4e0c-ae35-5b928760a6d8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:08.485868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 714/1441] 10.222.0.71 () {54 vars in 964 bytes} [Mon May 16 15:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:08.491747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18f8d1-1056-4fa4-aa8c-7315f9e38bc3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:08.492039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18f8d1-1056-4fa4-aa8c-7315f9e38bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:08.492316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18f8d1-1056-4fa4-aa8c-7315f9e38bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:08.528508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e18f8d1-1056-4fa4-aa8c-7315f9e38bc3 None None] MFA Rules not processed for user `76b2a2d381a14f5fb04e721905b36ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:08.529706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e18f8d1-1056-4fa4-aa8c-7315f9e38bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:08.536033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 728/1442] 10.222.0.71 () {58 vars in 1061 bytes} [Mon May 16 15:08:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:08.545505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de745b-8d96-4e0c-ae35-5b928760a6d8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:08.555743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de745b-8d96-4e0c-ae35-5b928760a6d8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:08.559150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d970dc-7a58-4b3b-8030-291e754a5704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:08.559811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d970dc-7a58-4b3b-8030-291e754a5704 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:08.560098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d970dc-7a58-4b3b-8030-291e754a5704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:08.560873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d970dc-7a58-4b3b-8030-291e754a5704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:08.569880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d970dc-7a58-4b3b-8030-291e754a5704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:08.571361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d970dc-7a58-4b3b-8030-291e754a5704 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:08.581098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 715/1443] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:11.366402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb94395-7464-4629-8eec-e372ccdb8e13 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:11.366898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb94395-7464-4629-8eec-e372ccdb8e13 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:11.367015 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb94395-7464-4629-8eec-e372ccdb8e13 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:11.368319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:11.373730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c641c9-81b8-48fb-b8db-986c00c3d0a6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:11.374113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c641c9-81b8-48fb-b8db-986c00c3d0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:11.374279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c641c9-81b8-48fb-b8db-986c00c3d0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:11.391405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-87c641c9-81b8-48fb-b8db-986c00c3d0a6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:11.392626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-87c641c9-81b8-48fb-b8db-986c00c3d0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:11.400196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:11.438378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb94395-7464-4629-8eec-e372ccdb8e13 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:11.448701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb94395-7464-4629-8eec-e372ccdb8e13 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:11.452315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d266-99ce-4ed5-beff-92d95c69c211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:11.452925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d266-99ce-4ed5-beff-92d95c69c211 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:11.453204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d266-99ce-4ed5-beff-92d95c69c211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:11.453476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d266-99ce-4ed5-beff-92d95c69c211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:11.462018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d266-99ce-4ed5-beff-92d95c69c211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:11.463429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d266-99ce-4ed5-beff-92d95c69c211 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:11.473182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 730/1446] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:14.136763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229130d6-efd6-4e19-8eb9-0d54947cb2b6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:14.137387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229130d6-efd6-4e19-8eb9-0d54947cb2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:14.137387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229130d6-efd6-4e19-8eb9-0d54947cb2b6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:14.138851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:14.144283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463583f-d7dd-4ee9-8787-e4de3e0c87b1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:14.144752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463583f-d7dd-4ee9-8787-e4de3e0c87b1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:14.144857 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463583f-d7dd-4ee9-8787-e4de3e0c87b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:14.162493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f463583f-d7dd-4ee9-8787-e4de3e0c87b1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:14.163701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f463583f-d7dd-4ee9-8787-e4de3e0c87b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:14.172719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:14.210988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229130d6-efd6-4e19-8eb9-0d54947cb2b6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:14.222107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229130d6-efd6-4e19-8eb9-0d54947cb2b6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:14.225518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb9442-a871-44fc-97ff-7e933a13a9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:14.226147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb9442-a871-44fc-97ff-7e933a13a9fd service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:14.226430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb9442-a871-44fc-97ff-7e933a13a9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:14.226703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb9442-a871-44fc-97ff-7e933a13a9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:14.236131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb9442-a871-44fc-97ff-7e933a13a9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:14.237545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb9442-a871-44fc-97ff-7e933a13a9fd service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:14.246573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 718/1449] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:16.589214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2432-ad41-401e-876e-4e57975f498f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:16.589794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2432-ad41-401e-876e-4e57975f498f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:16.589794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2432-ad41-401e-876e-4e57975f498f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:16.591917 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:16.597245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93424-b3c4-49d5-8398-c832ea5e4403 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:16.597658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93424-b3c4-49d5-8398-c832ea5e4403 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:16.597787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93424-b3c4-49d5-8398-c832ea5e4403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:16.615287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6be93424-b3c4-49d5-8398-c832ea5e4403 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:16.616481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6be93424-b3c4-49d5-8398-c832ea5e4403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:16.624213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:16.661707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2432-ad41-401e-876e-4e57975f498f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:16.673315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2432-ad41-401e-876e-4e57975f498f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:16.676787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d1cb1-8bbb-4c7a-9e62-2fa4c92ddc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:16.677422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d1cb1-8bbb-4c7a-9e62-2fa4c92ddc1a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:16.677708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d1cb1-8bbb-4c7a-9e62-2fa4c92ddc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:16.677988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d1cb1-8bbb-4c7a-9e62-2fa4c92ddc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:16.686912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d1cb1-8bbb-4c7a-9e62-2fa4c92ddc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:16.688363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d1cb1-8bbb-4c7a-9e62-2fa4c92ddc1a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:16.697518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 733/1452] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:18.989651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6906f8-9c32-4f92-922e-b125b315c30a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:18.990587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6906f8-9c32-4f92-922e-b125b315c30a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:18.990587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6906f8-9c32-4f92-922e-b125b315c30a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:18.991600 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:18.996815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953409f-ce2c-4d8e-a907-8e2a54cfdbca None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:18.997249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953409f-ce2c-4d8e-a907-8e2a54cfdbca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:18.997662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953409f-ce2c-4d8e-a907-8e2a54cfdbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:19.015281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6953409f-ce2c-4d8e-a907-8e2a54cfdbca None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:19.016504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6953409f-ce2c-4d8e-a907-8e2a54cfdbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:19.024365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:19.063692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6906f8-9c32-4f92-922e-b125b315c30a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:19.073836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6906f8-9c32-4f92-922e-b125b315c30a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:19.078056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95d2b5-27bb-42a2-8da0-ff39f45bb52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:19.078678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d2b5-27bb-42a2-8da0-ff39f45bb52f service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:19.078955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d2b5-27bb-42a2-8da0-ff39f45bb52f service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:19.079229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d2b5-27bb-42a2-8da0-ff39f45bb52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:19.087932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95d2b5-27bb-42a2-8da0-ff39f45bb52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:19.089335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95d2b5-27bb-42a2-8da0-ff39f45bb52f service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:19.098586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 721/1455] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:21.373136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833d290-bb49-4c72-b126-c243e588d4d2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:21.373700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833d290-bb49-4c72-b126-c243e588d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:21.373700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833d290-bb49-4c72-b126-c243e588d4d2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:21.375061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:21.380387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c895c-4710-47ae-8f5a-fd4de967f758 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:21.380785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c895c-4710-47ae-8f5a-fd4de967f758 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:21.380943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c895c-4710-47ae-8f5a-fd4de967f758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:21.398693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3c1c895c-4710-47ae-8f5a-fd4de967f758 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:21.399916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3c1c895c-4710-47ae-8f5a-fd4de967f758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:21.408312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:21.446193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833d290-bb49-4c72-b126-c243e588d4d2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:21.456955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833d290-bb49-4c72-b126-c243e588d4d2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:21.460447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9939b-dbac-4fc5-99be-f8489cbff254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:21.461084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9939b-dbac-4fc5-99be-f8489cbff254 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:21.461367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9939b-dbac-4fc5-99be-f8489cbff254 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:21.461642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9939b-dbac-4fc5-99be-f8489cbff254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:21.470275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9939b-dbac-4fc5-99be-f8489cbff254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:21.471763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9939b-dbac-4fc5-99be-f8489cbff254 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:21.481628 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 736/1458] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:23.808823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20294188-6524-4578-95ee-353b99e59eb6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:23.809387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20294188-6524-4578-95ee-353b99e59eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:23.809818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20294188-6524-4578-95ee-353b99e59eb6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:23.811260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:23.816878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596705d1-c18e-43ed-96af-ca59fc690783 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:23.817333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596705d1-c18e-43ed-96af-ca59fc690783 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:23.817444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596705d1-c18e-43ed-96af-ca59fc690783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:23.835084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-596705d1-c18e-43ed-96af-ca59fc690783 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:23.836314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-596705d1-c18e-43ed-96af-ca59fc690783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:23.844785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:23.880548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20294188-6524-4578-95ee-353b99e59eb6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:23.891402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20294188-6524-4578-95ee-353b99e59eb6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:23.894859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b43496-0b4c-427f-b442-0c754ff058ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:23.895490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43496-0b4c-427f-b442-0c754ff058ff service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:23.895812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43496-0b4c-427f-b442-0c754ff058ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:23.896089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43496-0b4c-427f-b442-0c754ff058ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:23.905425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b43496-0b4c-427f-b442-0c754ff058ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:23.906865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b43496-0b4c-427f-b442-0c754ff058ff service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:23.916480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 724/1461] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:26.187080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db17267-354f-4d6c-8688-14dde42e138e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:26.187608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db17267-354f-4d6c-8688-14dde42e138e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:26.187608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db17267-354f-4d6c-8688-14dde42e138e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:26.188953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:26.194351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c3d0-c5d6-4cf1-99cd-76338e63e4bc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:26.194761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c3d0-c5d6-4cf1-99cd-76338e63e4bc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:26.194908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c3d0-c5d6-4cf1-99cd-76338e63e4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:26.212952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-50f8c3d0-c5d6-4cf1-99cd-76338e63e4bc None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:26.214138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-50f8c3d0-c5d6-4cf1-99cd-76338e63e4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:26.222366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:26.258290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db17267-354f-4d6c-8688-14dde42e138e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:26.269193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db17267-354f-4d6c-8688-14dde42e138e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:26.272626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f1ba0-a721-4e00-a1b0-5ba8f8a58fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:26.273261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f1ba0-a721-4e00-a1b0-5ba8f8a58fde service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:26.273539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f1ba0-a721-4e00-a1b0-5ba8f8a58fde service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:26.273811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f1ba0-a721-4e00-a1b0-5ba8f8a58fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:26.282409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f1ba0-a721-4e00-a1b0-5ba8f8a58fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:26.283860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f1ba0-a721-4e00-a1b0-5ba8f8a58fde service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:26.293214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 739/1464] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:28.520092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d54e-8810-4bb7-923f-fc6236a9396a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:28.520709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d54e-8810-4bb7-923f-fc6236a9396a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:28.520709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d54e-8810-4bb7-923f-fc6236a9396a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:28.521990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:28.527783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88781652-4fd3-442f-8f87-3a14cccde822 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:28.528226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88781652-4fd3-442f-8f87-3a14cccde822 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:28.528491 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88781652-4fd3-442f-8f87-3a14cccde822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:28.547289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-88781652-4fd3-442f-8f87-3a14cccde822 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:28.548596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-88781652-4fd3-442f-8f87-3a14cccde822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:28.556267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:28.592426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2d54e-8810-4bb7-923f-fc6236a9396a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:28.602645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2d54e-8810-4bb7-923f-fc6236a9396a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:28.607272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947d89a-0bab-422a-a075-e6067de5d5ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:28.607936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947d89a-0bab-422a-a075-e6067de5d5ef service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:28.608223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947d89a-0bab-422a-a075-e6067de5d5ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:28.608494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947d89a-0bab-422a-a075-e6067de5d5ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:28.617644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947d89a-0bab-422a-a075-e6067de5d5ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:28.619075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947d89a-0bab-422a-a075-e6067de5d5ef service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:28.629893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 727/1467] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:08:30.939127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e587b-3733-4efc-a8a1-014f7234314d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:30.939722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e587b-3733-4efc-a8a1-014f7234314d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:30.939722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e587b-3733-4efc-a8a1-014f7234314d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:30.941076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:08:30.946273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e882f-d27c-4af1-9020-b90a46f90588 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:30.946714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e882f-d27c-4af1-9020-b90a46f90588 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:30.946867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e882f-d27c-4af1-9020-b90a46f90588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:30.965585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8b1e882f-d27c-4af1-9020-b90a46f90588 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:08:30.966904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8b1e882f-d27c-4af1-9020-b90a46f90588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:08:30.974464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:08:31.010170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e587b-3733-4efc-a8a1-014f7234314d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:08:31.021518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e587b-3733-4efc-a8a1-014f7234314d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:08:31.024960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de449fb3-8bfe-4fe7-b25d-217f08c1d356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:08:31.025589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de449fb3-8bfe-4fe7-b25d-217f08c1d356 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:08:31.025870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de449fb3-8bfe-4fe7-b25d-217f08c1d356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:08:31.026146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de449fb3-8bfe-4fe7-b25d-217f08c1d356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:08:31.035291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de449fb3-8bfe-4fe7-b25d-217f08c1d356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:08:31.036741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de449fb3-8bfe-4fe7-b25d-217f08c1d356 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:08:31.046770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 742/1470] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:08:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:09:04.312293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c32e7-5cb2-472b-a9f0-daa68f50f48a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:04.312895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c32e7-5cb2-472b-a9f0-daa68f50f48a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:04.312895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c32e7-5cb2-472b-a9f0-daa68f50f48a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:04.314180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 729/1471] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:04.319158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a23ad-b340-4191-868b-b7b81c279dd3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:04.319568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a23ad-b340-4191-868b-b7b81c279dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:04.319743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a23ad-b340-4191-868b-b7b81c279dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:04.338487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-815a23ad-b340-4191-868b-b7b81c279dd3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:04.339709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-815a23ad-b340-4191-868b-b7b81c279dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:04.347364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 743/1472] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:04.393037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84d039-dd7f-4eed-87fa-412f09a7eaeb None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:04.393317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84d039-dd7f-4eed-87fa-412f09a7eaeb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:04.393587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84d039-dd7f-4eed-87fa-412f09a7eaeb None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:04.394927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 730/1473] 10.222.0.71 () {54 vars in 962 bytes} [Mon May 16 15:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:04.400736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd89f-eb17-4a6a-830e-8595fd48f774 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:04.401031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd89f-eb17-4a6a-830e-8595fd48f774 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:04.401310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd89f-eb17-4a6a-830e-8595fd48f774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:04.419137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c28dd89f-eb17-4a6a-830e-8595fd48f774 None None] MFA Rules not processed for user `b60d4a75e5b8464a8c7935e7433cf170`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:04.420343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c28dd89f-eb17-4a6a-830e-8595fd48f774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:04.426599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 744/1474] 10.222.0.71 () {58 vars in 1059 bytes} [Mon May 16 15:09:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:04.433973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84d039-dd7f-4eed-87fa-412f09a7eaeb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:09:04.444053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84d039-dd7f-4eed-87fa-412f09a7eaeb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:09:04.448029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487b964-e639-4510-b115-6b00ed886c05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:09:04.448648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b964-e639-4510-b115-6b00ed886c05 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:04.448949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b964-e639-4510-b115-6b00ed886c05 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:04.449216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b964-e639-4510-b115-6b00ed886c05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:04.457643 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487b964-e639-4510-b115-6b00ed886c05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:09:04.459052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487b964-e639-4510-b115-6b00ed886c05 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:09:04.468804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 731/1475] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:09:06.771141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61792b7-51eb-4349-aaa9-aeb83e76495f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:06.771732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61792b7-51eb-4349-aaa9-aeb83e76495f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:06.771732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61792b7-51eb-4349-aaa9-aeb83e76495f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:06.773088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 745/1476] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:06.778856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069e4eb-5dda-4d4e-b241-f0c1e0bc19e5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:06.779275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069e4eb-5dda-4d4e-b241-f0c1e0bc19e5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:06.779411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069e4eb-5dda-4d4e-b241-f0c1e0bc19e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:06.797597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6069e4eb-5dda-4d4e-b241-f0c1e0bc19e5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:06.798769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6069e4eb-5dda-4d4e-b241-f0c1e0bc19e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:06.806279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 732/1477] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:06.828869 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61792b7-51eb-4349-aaa9-aeb83e76495f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:09:06.841798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61792b7-51eb-4349-aaa9-aeb83e76495f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:09:06.846350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31abf05f-6483-439f-9425-99f203c1ecae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:09:06.846999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31abf05f-6483-439f-9425-99f203c1ecae admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:06.847283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31abf05f-6483-439f-9425-99f203c1ecae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:06.847561 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31abf05f-6483-439f-9425-99f203c1ecae admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:06.848323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31abf05f-6483-439f-9425-99f203c1ecae admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:09:06.850096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-31abf05f-6483-439f-9425-99f203c1ecae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:09:06.850096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:09:06.850096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:09:06.851244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31abf05f-6483-439f-9425-99f203c1ecae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:09:06.870699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 746/1478] 10.222.0.71 () {56 vars in 1108 bytes} [Mon May 16 15:09:06 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:09:09.136864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d505db5-bd46-48ea-b3a2-21968783f31c None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:09.137405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d505db5-bd46-48ea-b3a2-21968783f31c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:09.137405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d505db5-bd46-48ea-b3a2-21968783f31c None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:09.138720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 733/1479] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:09.143903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9b831-ea5b-4c95-ab7f-d64148928cb3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:09.144346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9b831-ea5b-4c95-ab7f-d64148928cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:09.144457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9b831-ea5b-4c95-ab7f-d64148928cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:09.162607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6cd9b831-ea5b-4c95-ab7f-d64148928cb3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:09.163839 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6cd9b831-ea5b-4c95-ab7f-d64148928cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:09.172215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 747/1480] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:09.213617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d505db5-bd46-48ea-b3a2-21968783f31c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:09:09.227887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d505db5-bd46-48ea-b3a2-21968783f31c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:09:09.233151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474aab2c-716d-45cf-a2bd-25a2b9c2dc44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:09:09.233780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aab2c-716d-45cf-a2bd-25a2b9c2dc44 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:09.234056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aab2c-716d-45cf-a2bd-25a2b9c2dc44 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:09.234326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aab2c-716d-45cf-a2bd-25a2b9c2dc44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:09.242208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474aab2c-716d-45cf-a2bd-25a2b9c2dc44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:09:09.243816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474aab2c-716d-45cf-a2bd-25a2b9c2dc44 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:09:09.252880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 734/1481] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:09:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:09:11.660584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba03f-6cf4-4e33-b8d9-f8af177057b4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:11.661119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba03f-6cf4-4e33-b8d9-f8af177057b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:11.661119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba03f-6cf4-4e33-b8d9-f8af177057b4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:11.662386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 748/1482] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:11.668158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a706-1611-48ca-b87e-4db8206fcfe8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:11.668587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a706-1611-48ca-b87e-4db8206fcfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:11.668692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a706-1611-48ca-b87e-4db8206fcfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:11.686523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-75c5a706-1611-48ca-b87e-4db8206fcfe8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:11.687750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-75c5a706-1611-48ca-b87e-4db8206fcfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:11.695743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 735/1483] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:11.731305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba03f-6cf4-4e33-b8d9-f8af177057b4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:09:11.741827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba03f-6cf4-4e33-b8d9-f8af177057b4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:09:11.746096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b7ccb-250f-4af8-a875-ee29a2023e63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:09:11.746726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b7ccb-250f-4af8-a875-ee29a2023e63 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:11.747007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b7ccb-250f-4af8-a875-ee29a2023e63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:11.747281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b7ccb-250f-4af8-a875-ee29a2023e63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:11.756113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b7ccb-250f-4af8-a875-ee29a2023e63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:09:11.757530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b7ccb-250f-4af8-a875-ee29a2023e63 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:09:11.766751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 749/1484] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:09:14.051735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03639-b3e6-4e52-8c74-f07068c15acd None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:14.052294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03639-b3e6-4e52-8c74-f07068c15acd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:14.052294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03639-b3e6-4e52-8c74-f07068c15acd None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:14.053698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 736/1485] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:14.058832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1437f8a-db5a-4dd0-827d-9128272c8ebe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:14.059254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1437f8a-db5a-4dd0-827d-9128272c8ebe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:14.059390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1437f8a-db5a-4dd0-827d-9128272c8ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:14.078055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e1437f8a-db5a-4dd0-827d-9128272c8ebe None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:14.079240 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e1437f8a-db5a-4dd0-827d-9128272c8ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:14.087902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 750/1486] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:14.124510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03639-b3e6-4e52-8c74-f07068c15acd None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:09:14.135581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03639-b3e6-4e52-8c74-f07068c15acd None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:09:14.139045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404e8a8-4436-4da0-a173-80622d0834ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:09:14.139697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e8a8-4436-4da0-a173-80622d0834ea service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:14.139979 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e8a8-4436-4da0-a173-80622d0834ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:14.140252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e8a8-4436-4da0-a173-80622d0834ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:14.149711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404e8a8-4436-4da0-a173-80622d0834ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:09:14.151136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404e8a8-4436-4da0-a173-80622d0834ea service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:09:14.160474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 737/1487] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:09:16.625849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144df91-5893-4fa9-a5ce-c13204894801 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:16.626451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144df91-5893-4fa9-a5ce-c13204894801 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:16.626451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144df91-5893-4fa9-a5ce-c13204894801 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:16.627783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 751/1488] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:16.633928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25094b-60d6-4f4e-b445-701dc6a4966c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:16.634390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25094b-60d6-4f4e-b445-701dc6a4966c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:16.634505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25094b-60d6-4f4e-b445-701dc6a4966c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:16.651544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7a25094b-60d6-4f4e-b445-701dc6a4966c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:16.652747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7a25094b-60d6-4f4e-b445-701dc6a4966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:16.660795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 738/1489] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:16.701563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144df91-5893-4fa9-a5ce-c13204894801 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:09:16.712339 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144df91-5893-4fa9-a5ce-c13204894801 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:09:16.715874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589444a-dc35-432c-a52e-552d64bd5182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:09:16.716505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589444a-dc35-432c-a52e-552d64bd5182 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:16.716794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589444a-dc35-432c-a52e-552d64bd5182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:16.717071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589444a-dc35-432c-a52e-552d64bd5182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:16.725655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589444a-dc35-432c-a52e-552d64bd5182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:09:16.727083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589444a-dc35-432c-a52e-552d64bd5182 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:09:16.736197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 752/1490] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:09:19.318747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3375e76-af17-4580-b1f0-7195e5c3f775 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:19.319314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3375e76-af17-4580-b1f0-7195e5c3f775 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:19.319314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3375e76-af17-4580-b1f0-7195e5c3f775 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:19.320877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 739/1491] 10.222.0.71 () {54 vars in 926 bytes} [Mon May 16 15:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:09:19.326191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddb37d-023d-46cb-a860-06a1bfbaa633 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:19.326640 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddb37d-023d-46cb-a860-06a1bfbaa633 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:19.326751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddb37d-023d-46cb-a860-06a1bfbaa633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:19.344854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-68ddb37d-023d-46cb-a860-06a1bfbaa633 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:09:19.346116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-68ddb37d-023d-46cb-a860-06a1bfbaa633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:09:19.354699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 753/1492] 10.222.0.71 () {58 vars in 1023 bytes} [Mon May 16 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:09:19.392841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3375e76-af17-4580-b1f0-7195e5c3f775 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:09:19.403031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3375e76-af17-4580-b1f0-7195e5c3f775 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:09:19.406481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247cfe8-2976-4458-b815-4fcc24aad528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:09:19.407075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247cfe8-2976-4458-b815-4fcc24aad528 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:09:19.407351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247cfe8-2976-4458-b815-4fcc24aad528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:09:19.407618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247cfe8-2976-4458-b815-4fcc24aad528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:09:19.417041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247cfe8-2976-4458-b815-4fcc24aad528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:09:19.418455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247cfe8-2976-4458-b815-4fcc24aad528 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:09:19.427717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 740/1493] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:11:17.209458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5e90c-443d-48de-8648-106d7a1a5b1e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.210123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5e90c-443d-48de-8648-106d7a1a5b1e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.210123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5e90c-443d-48de-8648-106d7a1a5b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.227616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-78c5e90c-443d-48de-8648-106d7a1a5b1e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:11:17.228824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78c5e90c-443d-48de-8648-106d7a1a5b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:11:17.236465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 754/1494] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:11:17.246177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b237cf-4fd3-47e3-a185-76cea2fbc688 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.246679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b237cf-4fd3-47e3-a185-76cea2fbc688 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.246785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b237cf-4fd3-47e3-a185-76cea2fbc688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.264384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b2b237cf-4fd3-47e3-a185-76cea2fbc688 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:11:17.265550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b2b237cf-4fd3-47e3-a185-76cea2fbc688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:11:17.273102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 741/1495] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:11:17.277997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5e90c-443d-48de-8648-106d7a1a5b1e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:17.287423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5e90c-443d-48de-8648-106d7a1a5b1e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:17.291913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:17.292563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.292846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.293119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.293844 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:11:17.294149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:17.295162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:17.295162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:11:17.295162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:11:17.296257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:17.308138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 755/1496] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:11:17.316815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b237cf-4fd3-47e3-a185-76cea2fbc688 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:17.325452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b237cf-4fd3-47e3-a185-76cea2fbc688 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:17.329807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:17.330439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.330802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.331064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.331833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:17.333534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:17.333534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 16 15:11:17.333534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: May 16 15:11:17.334701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:17.451520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 742/1497] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:11:17 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:11:17.456281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:17.466483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16f008-6819-4395-b233-72b522a8dd39 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:17.521561 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:17.522233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.522518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.522794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.523487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:17.524571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:17.524571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:11:17.524571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:11:17.525779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:17.668488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 756/1498] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:11:17 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:11:17.672565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:17.683343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceeedf-8cef-44c1-a652-743d9a8dcb21 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:17.687845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:17.688515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.688798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.689068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.689800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:17.691149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:17.691149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 16 15:11:17.691149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: May 16 15:11:17.692064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:17.705110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 743/1499] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:11:17.709642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:17.720847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee509-5ae4-48fa-89db-a18c34a95d0f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:17.727107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab36efc-5813-4898-ac07-780b59c4861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:17.727791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36efc-5813-4898-ac07-780b59c4861f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.728070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36efc-5813-4898-ac07-780b59c4861f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.728342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab36efc-5813-4898-ac07-780b59c4861f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.729055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab36efc-5813-4898-ac07-780b59c4861f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:17.730132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-dab36efc-5813-4898-ac07-780b59c4861f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:17.730132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:11:17.730132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:11:17.731189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab36efc-5813-4898-ac07-780b59c4861f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:17.745629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 757/1500] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:11:17.749782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:17.759934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c00396-c25e-4008-91c7-1b5207a9023c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:17.764349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c85f51-7fc1-4c1f-8e5a-deafa89c390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:17.765219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85f51-7fc1-4c1f-8e5a-deafa89c390c admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.765498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85f51-7fc1-4c1f-8e5a-deafa89c390c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.765771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85f51-7fc1-4c1f-8e5a-deafa89c390c admin admin] PATH_INFO: `/v3/projects/1f9e8f3b111e4b93907b97d9ef967cd1/users/8e3a2bcf16fa4753837099faf34a9b1a/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.779849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c85f51-7fc1-4c1f-8e5a-deafa89c390c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f9e8f3b111e4b93907b97d9ef967cd1, user_id=8e3a2bcf16fa4753837099faf34a9b1a, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:17.781043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-97c85f51-7fc1-4c1f-8e5a-deafa89c390c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:17.781043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 16 15:11:17.781043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: May 16 15:11:17.782820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c85f51-7fc1-4c1f-8e5a-deafa89c390c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:17.862439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 744/1501] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:11:17 2022] PUT /identity/v3/projects/1f9e8f3b111e4b93907b97d9ef967cd1/users/8e3a2bcf16fa4753837099faf34a9b1a/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:11:17.874112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0119af3-685e-4ce1-8d6e-c72611f8d537 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.874394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0119af3-685e-4ce1-8d6e-c72611f8d537 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.874665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0119af3-685e-4ce1-8d6e-c72611f8d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.900625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b0119af3-685e-4ce1-8d6e-c72611f8d537 None None] MFA Rules not processed for user `8e3a2bcf16fa4753837099faf34a9b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:11:17.901778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b0119af3-685e-4ce1-8d6e-c72611f8d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:11:17.964351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 758/1502] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:11:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:11:17.977977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4f3e-439a-4de4-9459-4f2674511135 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.978209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4f3e-439a-4de4-9459-4f2674511135 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.978488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4f3e-439a-4de4-9459-4f2674511135 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:17.979882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 745/1503] 10.222.0.71 () {54 vars in 963 bytes} [Mon May 16 15:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:11:17.984494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa58eb-77e6-4b7d-924e-e3a5b8f87bac None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:17.984780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa58eb-77e6-4b7d-924e-e3a5b8f87bac None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:17.985052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa58eb-77e6-4b7d-924e-e3a5b8f87bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:18.001978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8aaa58eb-77e6-4b7d-924e-e3a5b8f87bac None None] MFA Rules not processed for user `1f2421c1bfb5402da3179e6f3c749374`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:11:18.003270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8aaa58eb-77e6-4b7d-924e-e3a5b8f87bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:11:18.051573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 759/1504] 10.222.0.71 () {58 vars in 1060 bytes} [Mon May 16 15:11:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:11:18.058713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4f3e-439a-4de4-9459-4f2674511135 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:18.069094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4f3e-439a-4de4-9459-4f2674511135 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:18.072499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:18.073119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:18.073413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:18.073670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:18.082382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:18.083901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:18.092277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 746/1505] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:11:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:11:18.134892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5989138-f373-4b75-a5af-a5a21809606c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:18.135180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5989138-f373-4b75-a5af-a5a21809606c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:18.135455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5989138-f373-4b75-a5af-a5a21809606c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:18.136879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 760/1506] 10.222.0.71 () {52 vars in 829 bytes} [Mon May 16 15:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:11:18.140341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:18.199941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb24f36-d680-4578-ae58-9346f50a1f14 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:18.204240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:18.205091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:18.205363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:18.206181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] PATH_INFO: `/v3/users/8e3a2bcf16fa4753837099faf34a9b1a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:18.207333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3a2bcf16fa4753837099faf34a9b1a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:18.208373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:18.208373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 16 15:11:18.208373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: May 16 15:11:18.209564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:18.411693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3a2bcf16fa4753837099faf34a9b1a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:18.473061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3a2bcf16fa4753837099faf34a9b1a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:18.473698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3a2bcf16fa4753837099faf34a9b1a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:18.493259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3a2bcf16fa4753837099faf34a9b1a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:18.505700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3a2bcf16fa4753837099faf34a9b1a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:18.511895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 747/1507] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:11:18 2022] DELETE /identity/v3/users/8e3a2bcf16fa4753837099faf34a9b1a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:11:18.517233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:18.517233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:18.517233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:18.535051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:11:18.536258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:11:18.605828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 761/1508] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:11:18.615843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:18.617282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:11:18.672941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8963f41-abb9-4439-a741-39e6c739e427 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:18.677067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0b349-f895-4be6-a32f-d29f17bb5ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:18.677707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0b349-f895-4be6-a32f-d29f17bb5ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:18.677991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0b349-f895-4be6-a32f-d29f17bb5ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:18.678269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0b349-f895-4be6-a32f-d29f17bb5ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:18.686884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0b349-f895-4be6-a32f-d29f17bb5ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:18.688351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0b349-f895-4be6-a32f-d29f17bb5ea6 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:18.698162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 748/1509] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:11:19.167226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:11:19.168677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:11:19.181091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2b09-d650-41bf-b0b5-02f3364357e7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:11:19.185497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:11:19.188308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:11:19.188595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:11:19.188867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] PATH_INFO: `/v3/projects/1f9e8f3b111e4b93907b97d9ef967cd1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:11:19.190338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f9e8f3b111e4b93907b97d9ef967cd1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:11:19.192290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:11:19.192290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:11:19.192290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:11:19.193850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:11:19.307959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f9e8f3b111e4b93907b97d9ef967cd1', 'request_id': 'req-0ca15534-281a-4820-ac3e-db213636fadc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:19.313883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f9e8f3b111e4b93907b97d9ef967cd1', 'request_id': 'req-0ca15534-281a-4820-ac3e-db213636fadc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:19.327049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f9e8f3b111e4b93907b97d9ef967cd1', 'request_id': 'req-0ca15534-281a-4820-ac3e-db213636fadc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:19.415855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] The token cache is being invalidate because project 1f9e8f3b111e4b93907b97d9ef967cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:11:19.416673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:19.417079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0ca15534-281a-4820-ac3e-db213636fadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:11:19.418618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 762/1510] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:11:19 2022] DELETE /identity/v3/projects/1f9e8f3b111e4b93907b97d9ef967cd1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:13:05.456835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541f073-3d20-4edc-acb2-38e88ff7d35f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:05.457401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541f073-3d20-4edc-acb2-38e88ff7d35f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:05.457401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541f073-3d20-4edc-acb2-38e88ff7d35f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:05.459489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:05.465567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf17fec-c131-4d48-a561-8bea63acc0e4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:05.469168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf17fec-c131-4d48-a561-8bea63acc0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:05.469168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf17fec-c131-4d48-a561-8bea63acc0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:05.484567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cbf17fec-c131-4d48-a561-8bea63acc0e4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:05.485749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cbf17fec-c131-4d48-a561-8bea63acc0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:05.546456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:05.713464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541f073-3d20-4edc-acb2-38e88ff7d35f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:05.715040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4541f073-3d20-4edc-acb2-38e88ff7d35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:05.786006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541f073-3d20-4edc-acb2-38e88ff7d35f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:05.791218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764a81c-41af-43e4-b47a-73190ef7fb89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:05.791885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a81c-41af-43e4-b47a-73190ef7fb89 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:05.792163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a81c-41af-43e4-b47a-73190ef7fb89 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:05.792434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a81c-41af-43e4-b47a-73190ef7fb89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:05.800338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764a81c-41af-43e4-b47a-73190ef7fb89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:05.801902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764a81c-41af-43e4-b47a-73190ef7fb89 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:05.810775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 750/1513] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:06.168149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d57d90-5a88-4fa3-bc5b-3d56b7de92a7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:06.168482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d57d90-5a88-4fa3-bc5b-3d56b7de92a7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:06.168856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d57d90-5a88-4fa3-bc5b-3d56b7de92a7 None None] PATH_INFO: `/v3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:06.170250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 764/1514] 10.222.0.25 () {54 vars in 925 bytes} [Mon May 16 15:13:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:13:06.176998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a4716-29b3-457e-b6a5-8912d9f479f2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:06.177292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a4716-29b3-457e-b6a5-8912d9f479f2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:06.177602 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a4716-29b3-457e-b6a5-8912d9f479f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:06.195789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1a2a4716-29b3-457e-b6a5-8912d9f479f2 None None] MFA Rules not processed for user `6c36906c508f41cca018e5f6dcaadd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:06.196993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1a2a4716-29b3-457e-b6a5-8912d9f479f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:06.262577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 751/1515] 10.222.0.25 () {58 vars in 1015 bytes} [Mon May 16 15:13:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:13:06.700845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034358-a1ef-4b6c-8ea9-4c3c18b058f3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:06.701486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034358-a1ef-4b6c-8ea9-4c3c18b058f3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:06.701486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034358-a1ef-4b6c-8ea9-4c3c18b058f3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:06.702756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 765/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:06.708115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d841d9a-76bb-428f-9909-d5b324cff59c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:06.708560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d841d9a-76bb-428f-9909-d5b324cff59c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:06.708705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d841d9a-76bb-428f-9909-d5b324cff59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:06.726843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2d841d9a-76bb-428f-9909-d5b324cff59c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:06.728060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2d841d9a-76bb-428f-9909-d5b324cff59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:06.735610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 752/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:06.885754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea034358-a1ef-4b6c-8ea9-4c3c18b058f3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:06.900137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea034358-a1ef-4b6c-8ea9-4c3c18b058f3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:06.905395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6483569-6315-4991-ba8a-a9706888f884 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:06.906031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483569-6315-4991-ba8a-a9706888f884 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:06.906310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483569-6315-4991-ba8a-a9706888f884 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:06.906584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483569-6315-4991-ba8a-a9706888f884 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:06.915184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6483569-6315-4991-ba8a-a9706888f884 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:06.916838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6483569-6315-4991-ba8a-a9706888f884 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:06.926936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 766/1518] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:15.509984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eb048-7990-4e17-ae02-c2b301dff6e5 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:15.510522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eb048-7990-4e17-ae02-c2b301dff6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:15.510522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eb048-7990-4e17-ae02-c2b301dff6e5 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:15.511929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 753/1519] 10.222.0.71 () {54 vars in 960 bytes} [Mon May 16 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:15.516923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cdecf-909d-45f9-a2a5-e30fed4a669d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:15.517385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cdecf-909d-45f9-a2a5-e30fed4a669d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:15.517487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cdecf-909d-45f9-a2a5-e30fed4a669d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:15.536516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-205cdecf-909d-45f9-a2a5-e30fed4a669d None None] MFA Rules not processed for user `6c36906c508f41cca018e5f6dcaadd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:15.537724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-205cdecf-909d-45f9-a2a5-e30fed4a669d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:15.546165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 767/1520] 10.222.0.71 () {58 vars in 1057 bytes} [Mon May 16 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:15.553761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5eb048-7990-4e17-ae02-c2b301dff6e5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:15.567206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5eb048-7990-4e17-ae02-c2b301dff6e5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:15.572055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1043c-5f70-4585-a3cb-9f61dd8d9af3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:15.572681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1043c-5f70-4585-a3cb-9f61dd8d9af3 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:15.572964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1043c-5f70-4585-a3cb-9f61dd8d9af3 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:15.573239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1043c-5f70-4585-a3cb-9f61dd8d9af3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:15.585914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1043c-5f70-4585-a3cb-9f61dd8d9af3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:15.587499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1043c-5f70-4585-a3cb-9f61dd8d9af3 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:15.596864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 754/1521] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:18.240497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a934aa8-a4f4-415f-bd5e-b84ba1568be9 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:18.241051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a934aa8-a4f4-415f-bd5e-b84ba1568be9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:18.241051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a934aa8-a4f4-415f-bd5e-b84ba1568be9 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:18.242433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:18.247206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1bd55-cea5-439e-8799-ad70ccc2c88b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:18.247676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1bd55-cea5-439e-8799-ad70ccc2c88b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:18.247793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1bd55-cea5-439e-8799-ad70ccc2c88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:18.266200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b6b1bd55-cea5-439e-8799-ad70ccc2c88b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:18.267411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b6b1bd55-cea5-439e-8799-ad70ccc2c88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:18.276450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:18.427005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a934aa8-a4f4-415f-bd5e-b84ba1568be9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:18.441215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a934aa8-a4f4-415f-bd5e-b84ba1568be9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:18.446466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230176c-0c8f-4ba2-b5f4-14f012ddb85a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:18.447093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230176c-0c8f-4ba2-b5f4-14f012ddb85a service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:18.447372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230176c-0c8f-4ba2-b5f4-14f012ddb85a service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:18.447671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230176c-0c8f-4ba2-b5f4-14f012ddb85a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:18.455452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230176c-0c8f-4ba2-b5f4-14f012ddb85a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:18.457673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230176c-0c8f-4ba2-b5f4-14f012ddb85a service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:18.466690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 769/1524] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:19.356226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172bf37-f7ac-4bc6-9e28-8c52cf1167d9 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:19.356802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172bf37-f7ac-4bc6-9e28-8c52cf1167d9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:19.356802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172bf37-f7ac-4bc6-9e28-8c52cf1167d9 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:19.358171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 15:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:19.364141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c18c5b-0626-40a8-89b4-f286c569b021 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:19.364551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c18c5b-0626-40a8-89b4-f286c569b021 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:19.364701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c18c5b-0626-40a8-89b4-f286c569b021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:19.382412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e1c18c5b-0626-40a8-89b4-f286c569b021 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:19.383617 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e1c18c5b-0626-40a8-89b4-f286c569b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:19.391602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:19.544518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172bf37-f7ac-4bc6-9e28-8c52cf1167d9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:19.558995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172bf37-f7ac-4bc6-9e28-8c52cf1167d9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:19.564300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a30f9-6ed8-4040-add8-84c54907fa7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:19.564935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a30f9-6ed8-4040-add8-84c54907fa7c service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:19.565310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a30f9-6ed8-4040-add8-84c54907fa7c service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:19.565587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a30f9-6ed8-4040-add8-84c54907fa7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:19.573500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a30f9-6ed8-4040-add8-84c54907fa7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:19.575070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a30f9-6ed8-4040-add8-84c54907fa7c service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:19.585492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 757/1527] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:23.975208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e478c4-00af-40e9-9a3e-a6b46c55a5a2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:23.975838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e478c4-00af-40e9-9a3e-a6b46c55a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:23.975838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e478c4-00af-40e9-9a3e-a6b46c55a5a2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:23.977175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:23.982314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205703d-522a-477e-acc9-da5dbd71ef6c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:23.982767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205703d-522a-477e-acc9-da5dbd71ef6c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:23.982876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205703d-522a-477e-acc9-da5dbd71ef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:24.000390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2205703d-522a-477e-acc9-da5dbd71ef6c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:24.001578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2205703d-522a-477e-acc9-da5dbd71ef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:24.009980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:24.047069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e478c4-00af-40e9-9a3e-a6b46c55a5a2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:24.048757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95e478c4-00af-40e9-9a3e-a6b46c55a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:24.102980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e478c4-00af-40e9-9a3e-a6b46c55a5a2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:24.106386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff79b-df85-4aa7-b8a8-c55aacd56c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:24.107017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff79b-df85-4aa7-b8a8-c55aacd56c11 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:24.107297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff79b-df85-4aa7-b8a8-c55aacd56c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:24.107568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff79b-df85-4aa7-b8a8-c55aacd56c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:24.116219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff79b-df85-4aa7-b8a8-c55aacd56c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:24.117637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff79b-df85-4aa7-b8a8-c55aacd56c11 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:24.126616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 772/1530] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:26.327243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023af0-eb6a-4eb8-a76e-08bd9f6157be None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:26.327857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023af0-eb6a-4eb8-a76e-08bd9f6157be None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:26.327857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023af0-eb6a-4eb8-a76e-08bd9f6157be None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:26.329192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:26.334985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4528e43-02c0-40c8-876e-fa3a10a47abe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:26.335428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4528e43-02c0-40c8-876e-fa3a10a47abe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:26.335586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4528e43-02c0-40c8-876e-fa3a10a47abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:26.353619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e4528e43-02c0-40c8-876e-fa3a10a47abe None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:26.354813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e4528e43-02c0-40c8-876e-fa3a10a47abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:26.362652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:26.398605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d023af0-eb6a-4eb8-a76e-08bd9f6157be None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:26.409811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d023af0-eb6a-4eb8-a76e-08bd9f6157be None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:26.413302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07f42-6518-44a5-8f4f-797409ce2d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:26.413931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07f42-6518-44a5-8f4f-797409ce2d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:26.414214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07f42-6518-44a5-8f4f-797409ce2d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:26.414495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07f42-6518-44a5-8f4f-797409ce2d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:26.423113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e07f42-6518-44a5-8f4f-797409ce2d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:26.425243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e07f42-6518-44a5-8f4f-797409ce2d7c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:26.434370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 760/1533] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:30.317548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20231b4-a903-40f1-b10c-16d42500d04a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:30.318117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20231b4-a903-40f1-b10c-16d42500d04a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:30.318117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20231b4-a903-40f1-b10c-16d42500d04a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:30.319460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:30.324649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d17187-d099-4fdf-8b6f-b9029dfa4c71 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:30.325060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d17187-d099-4fdf-8b6f-b9029dfa4c71 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:30.325199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d17187-d099-4fdf-8b6f-b9029dfa4c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:30.342769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-48d17187-d099-4fdf-8b6f-b9029dfa4c71 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:30.344012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-48d17187-d099-4fdf-8b6f-b9029dfa4c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:30.351936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:30.635984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20231b4-a903-40f1-b10c-16d42500d04a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:30.637643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b20231b4-a903-40f1-b10c-16d42500d04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:30.692632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20231b4-a903-40f1-b10c-16d42500d04a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:30.696150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2ee44-2af4-4c93-a2f8-5ffe5911010a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:30.696790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2ee44-2af4-4c93-a2f8-5ffe5911010a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:30.697065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2ee44-2af4-4c93-a2f8-5ffe5911010a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:30.697336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2ee44-2af4-4c93-a2f8-5ffe5911010a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:30.705960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2ee44-2af4-4c93-a2f8-5ffe5911010a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:30.707381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2ee44-2af4-4c93-a2f8-5ffe5911010a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:30.716456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 775/1536] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:13:32.978279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6023-d796-466f-a6b0-b775a7c758c3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:32.978834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6023-d796-466f-a6b0-b775a7c758c3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:32.978834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6023-d796-466f-a6b0-b775a7c758c3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:32.980186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:13:32.985291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b4a59-9b41-4703-ae0a-62d9ec51463d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:32.985731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b4a59-9b41-4703-ae0a-62d9ec51463d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:32.985841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b4a59-9b41-4703-ae0a-62d9ec51463d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:33.007890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f81b4a59-9b41-4703-ae0a-62d9ec51463d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:13:33.009085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f81b4a59-9b41-4703-ae0a-62d9ec51463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:33.016783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:13:33.285648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b6023-d796-466f-a6b0-b775a7c758c3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:13:33.287302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-af8b6023-d796-466f-a6b0-b775a7c758c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:13:33.300247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b6023-d796-466f-a6b0-b775a7c758c3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:13:33.303701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e81818-8ba0-4ae9-95b8-20a1586d28c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:13:33.304332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e81818-8ba0-4ae9-95b8-20a1586d28c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:13:33.304610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e81818-8ba0-4ae9-95b8-20a1586d28c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:13:33.304880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e81818-8ba0-4ae9-95b8-20a1586d28c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:13:33.313409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e81818-8ba0-4ae9-95b8-20a1586d28c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:13:33.315399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e81818-8ba0-4ae9-95b8-20a1586d28c3 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:13:33.324681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 763/1539] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:06.792810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee07982-ea1d-4c82-a14a-69bc08af727a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:06.793455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee07982-ea1d-4c82-a14a-69bc08af727a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:06.793455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee07982-ea1d-4c82-a14a-69bc08af727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:06.822152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9ee07982-ea1d-4c82-a14a-69bc08af727a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:06.823378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9ee07982-ea1d-4c82-a14a-69bc08af727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:06.837181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 777/1540] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:06.850877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02993590-a14f-44f6-a5a0-5e6c1c6d85dc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:06.851471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02993590-a14f-44f6-a5a0-5e6c1c6d85dc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:06.851471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02993590-a14f-44f6-a5a0-5e6c1c6d85dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:06.884724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-02993590-a14f-44f6-a5a0-5e6c1c6d85dc None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:06.885932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-02993590-a14f-44f6-a5a0-5e6c1c6d85dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:06.898527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 764/1541] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:06.903777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee07982-ea1d-4c82-a14a-69bc08af727a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:06.917090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee07982-ea1d-4c82-a14a-69bc08af727a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:06.928236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944bc13-8d3d-4537-ac6f-4fa738562178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:06.928920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944bc13-8d3d-4537-ac6f-4fa738562178 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:06.929212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944bc13-8d3d-4537-ac6f-4fa738562178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:06.929497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944bc13-8d3d-4537-ac6f-4fa738562178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:06.930561 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944bc13-8d3d-4537-ac6f-4fa738562178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:06.930862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944bc13-8d3d-4537-ac6f-4fa738562178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:06.932603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944bc13-8d3d-4537-ac6f-4fa738562178 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:06.958107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 778/1542] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:06.966733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02993590-a14f-44f6-a5a0-5e6c1c6d85dc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:06.981235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02993590-a14f-44f6-a5a0-5e6c1c6d85dc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:06.988664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4776a-c640-4f7c-bf33-aa4d4881ccc0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:06.989002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4776a-c640-4f7c-bf33-aa4d4881ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:06.989424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4776a-c640-4f7c-bf33-aa4d4881ccc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:06.991312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec57fb4-3915-4aba-854d-02f9e9dada62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:06.992029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec57fb4-3915-4aba-854d-02f9e9dada62 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:06.992385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec57fb4-3915-4aba-854d-02f9e9dada62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:06.992579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec57fb4-3915-4aba-854d-02f9e9dada62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:06.993274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec57fb4-3915-4aba-854d-02f9e9dada62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:06.995220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec57fb4-3915-4aba-854d-02f9e9dada62 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.024573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9da4776a-c640-4f7c-bf33-aa4d4881ccc0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.026643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9da4776a-c640-4f7c-bf33-aa4d4881ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.041798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 779/1543] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:07.043262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d143294-b002-43e7-abda-f63f6af5148f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.043549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d143294-b002-43e7-abda-f63f6af5148f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.043861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d143294-b002-43e7-abda-f63f6af5148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.072770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2d143294-b002-43e7-abda-f63f6af5148f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.073989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2d143294-b002-43e7-abda-f63f6af5148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.090154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 780/1544] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.091434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a083e-9b6b-4706-8831-dd39994964d1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.091752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a083e-9b6b-4706-8831-dd39994964d1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.092029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a083e-9b6b-4706-8831-dd39994964d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.121067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 765/1545] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:06 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:07.122418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a85a1-b9aa-4d15-8835-1dca3dde050c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.122701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a85a1-b9aa-4d15-8835-1dca3dde050c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.122971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a85a1-b9aa-4d15-8835-1dca3dde050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.129455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c47a083e-9b6b-4706-8831-dd39994964d1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.131449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c47a083e-9b6b-4706-8831-dd39994964d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.149457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-344a85a1-b9aa-4d15-8835-1dca3dde050c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.150638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-344a85a1-b9aa-4d15-8835-1dca3dde050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.206970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 781/1546] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.209661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cf6f0-6457-4383-9335-8da7898ac271 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.209661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cf6f0-6457-4383-9335-8da7898ac271 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.209661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cf6f0-6457-4383-9335-8da7898ac271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.233448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb6cf6f0-6457-4383-9335-8da7898ac271 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.234652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb6cf6f0-6457-4383-9335-8da7898ac271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.240247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 766/1547] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.241060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a85a1-b9aa-4d15-8835-1dca3dde050c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.243432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 782/1548] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.248062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c06ec7-90ce-43e1-b775-26410bb76e80 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.248353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c06ec7-90ce-43e1-b775-26410bb76e80 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.248626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c06ec7-90ce-43e1-b775-26410bb76e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.250799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a85a1-b9aa-4d15-8835-1dca3dde050c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.256629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.257658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.258130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.258560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.259401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.261161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:15:07.261161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 16 15:15:07.261161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: May 16 15:15:07.262494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.266033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-29c06ec7-90ce-43e1-b775-26410bb76e80 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.267213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-29c06ec7-90ce-43e1-b775-26410bb76e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.275943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 783/1549] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.277164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b38af0-b823-4a0e-9d54-293a96c60772 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.277451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b38af0-b823-4a0e-9d54-293a96c60772 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.277724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b38af0-b823-4a0e-9d54-293a96c60772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.295446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-52b38af0-b823-4a0e-9d54-293a96c60772 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.296660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-52b38af0-b823-4a0e-9d54-293a96c60772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.304985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 784/1550] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.306174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313db0-bad9-41bc-a2c7-1eb5d39e7f9d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.306459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313db0-bad9-41bc-a2c7-1eb5d39e7f9d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.306733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313db0-bad9-41bc-a2c7-1eb5d39e7f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.323666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ec313db0-bad9-41bc-a2c7-1eb5d39e7f9d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.324831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ec313db0-bad9-41bc-a2c7-1eb5d39e7f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.332674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 785/1551] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.333846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5cab9-c75b-45d1-b91e-386675014c11 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.340336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5cab9-c75b-45d1-b91e-386675014c11 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.340336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5cab9-c75b-45d1-b91e-386675014c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.351881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fea5cab9-c75b-45d1-b91e-386675014c11 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.353056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fea5cab9-c75b-45d1-b91e-386675014c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.360805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 786/1552] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.360995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 767/1553] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:07.361947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.362065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab83398-c66b-401c-9736-fecd68beefe9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.362287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab83398-c66b-401c-9736-fecd68beefe9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.362565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab83398-c66b-401c-9736-fecd68beefe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.374172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb4f5a-785d-4ec0-b49c-e09ee4a7cabd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.378841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.379871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.380163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.380550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.381287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:07.381594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.382605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 15:15:07.382605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 16 15:15:07.382605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: May 16 15:15:07.383479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675a1fe-55e7-4f73-9590-66433e0a3fe5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.383813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5ab83398-c66b-401c-9736-fecd68beefe9 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.384994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5ab83398-c66b-401c-9736-fecd68beefe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.392751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 787/1554] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.393529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab83398-c66b-401c-9736-fecd68beefe9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.402508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 768/1555] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:07.403929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab83398-c66b-401c-9736-fecd68beefe9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.404314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b842f23-5e3d-45e2-9f17-94a92ef34c0d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.404606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b842f23-5e3d-45e2-9f17-94a92ef34c0d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.404882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b842f23-5e3d-45e2-9f17-94a92ef34c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.408569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1074c62-9e24-4a28-9911-1905041c7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.409212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1074c62-9e24-4a28-9911-1905041c7293 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.409494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1074c62-9e24-4a28-9911-1905041c7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.409768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1074c62-9e24-4a28-9911-1905041c7293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.411091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1074c62-9e24-4a28-9911-1905041c7293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:07.411392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1074c62-9e24-4a28-9911-1905041c7293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.413027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1074c62-9e24-4a28-9911-1905041c7293 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.422869 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3b842f23-5e3d-45e2-9f17-94a92ef34c0d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.424098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3b842f23-5e3d-45e2-9f17-94a92ef34c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.424230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 788/1556] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:07.425536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f60a4-2a47-4931-9a29-7e879821b371 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.425882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f60a4-2a47-4931-9a29-7e879821b371 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.426156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f60a4-2a47-4931-9a29-7e879821b371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.437401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 769/1557] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.439053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0ee59-c105-4c83-b6d9-1a31d9a3049c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.439451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0ee59-c105-4c83-b6d9-1a31d9a3049c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.439757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0ee59-c105-4c83-b6d9-1a31d9a3049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.444230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f45f60a4-2a47-4931-9a29-7e879821b371 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.445401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f45f60a4-2a47-4931-9a29-7e879821b371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.453196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 789/1558] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.453959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f60a4-2a47-4931-9a29-7e879821b371 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.457833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-71b0ee59-c105-4c83-b6d9-1a31d9a3049c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.458953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-71b0ee59-c105-4c83-b6d9-1a31d9a3049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.463953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f60a4-2a47-4931-9a29-7e879821b371 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.466728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 770/1559] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.467961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d66cd-6641-4670-a5c5-79a2f9659f72 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.468247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d66cd-6641-4670-a5c5-79a2f9659f72 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.468518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d66cd-6641-4670-a5c5-79a2f9659f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.472581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.473778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.474055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.474327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.475043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.476686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.489433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 790/1560] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:07.489609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-827d66cd-6641-4670-a5c5-79a2f9659f72 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:07.489746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.489862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-827d66cd-6641-4670-a5c5-79a2f9659f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:07.496622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 771/1561] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:07.497208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4161c-b13b-47d4-b0ba-5e9ce7363405 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.497451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d66cd-6641-4670-a5c5-79a2f9659f72 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.504446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.505173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.505457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.505729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.506450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:07.506764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.507171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d66cd-6641-4670-a5c5-79a2f9659f72 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.508411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.511705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.512341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.512626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.512903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.513586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.515466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.521537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 791/1562] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:07.522694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.533133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c5197-b6f6-41ce-9401-929bbcf5b030 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.538977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.539660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.539947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.540226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.540930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.542847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:15:07.542847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:15:07.542847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:15:07.543144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.668468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 772/1563] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:07.668468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.697993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 792/1564] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:07.698894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.733444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e5c3c-601e-4f00-977f-a352dc652dcb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.759470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378f3b1-277c-433b-8000-a4cea6383c44 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.764149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.764808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.765086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.765358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.766075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:07.766387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.767997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.779890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 793/1565] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:07.780822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.789129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.789803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.790111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.790382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.790705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2cfe3-8db4-4250-85bc-333fe53a7246 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.791107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:07.791411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.793040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.795163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.795821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.796102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.796372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.797081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:07.797386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.798958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.805091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 773/1566] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:07.805908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.815353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861535b-0648-46e6-8330-0bf9703b3033 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.818170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 794/1567] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:07.819171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.820536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.821191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.821472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.821764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.822511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.824368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.828380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310898c8-aaca-40e8-8624-d781ef048434 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.833448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.834128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.834411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.834686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.837995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 774/1568] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:07.838143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:07.838143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.838352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.838432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.845478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a078d7-b0e7-44fa-9ab6-25d19d16f5cb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.853338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 795/1569] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:07.853338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.854529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.855204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.855470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.855753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.856537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.858426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.859758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54610ed0-1a43-485b-828a-5d7fe7a2d42a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.864891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:07.865534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:07.865818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:07.866098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:07.866769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:07.868707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:07.936817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 775/1570] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:07.937439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.963767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 796/1571] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:07.965274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:07.975174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197db26-a645-44cc-8452-e0a8050d89c8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:07.999819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396084f-92f3-4e6c-bcb0-cfd02b20a418 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.004645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.015704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.015704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.015704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.015704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.015704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.030782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.031447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.031768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.032046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.032735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.034692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.098678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 776/1572] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.099686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.108834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bebdd-82fd-4811-bd26-2e523a64bff8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.113886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.114597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.114873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.115142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.115836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.117688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.128065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 797/1573] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:07 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.128065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.188535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d130fa-0ea5-40ba-8686-11f9f2deaae7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.193258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.193931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.194208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.194476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.195145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.197073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.214352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 777/1574] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.215314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.273345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a3ec3-367a-4f73-bede-50ade3c5f2f3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.277790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.278661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.284876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.284876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] PATH_INFO: `/v3/projects/e1e340fabe31456cb021fcd0b5f505b5/users/97021ddaffff4e088961ba1873a8e697/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.285150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 798/1575] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.285150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.294680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e340fabe31456cb021fcd0b5f505b5, user_id=97021ddaffff4e088961ba1873a8e697, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.297642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.341576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9a8bf-8a62-430a-aef4-3c78db227359 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.346077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.346731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.347017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.347298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.348623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.350514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.380847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 778/1576] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:08 2022] PUT /identity/v3/projects/e1e340fabe31456cb021fcd0b5f505b5/users/97021ddaffff4e088961ba1873a8e697/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:08.381653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.431153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 799/1577] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.431153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.440451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7eb2-352b-4aca-bd96-9d7716f85cfe admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.489474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22472f7-b313-4fd7-b53a-2366c1cb371c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.494097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.494449 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.494782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.495071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.495232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.495351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.495454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.495668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.496097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.496355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.497687 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.498282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.507728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 800/1578] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.508518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.517542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16295741-5299-4545-b81a-ba624b5b1935 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.522196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.523145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.523476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.523726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.524422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.525967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.535872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 801/1579] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.536698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.545530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633491f-2543-4860-ae30-3b5dc308cb0e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.549918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.550516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.551588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.551909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.552562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.554412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.598404 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 779/1580] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.599263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.608353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da23e0-e778-46e7-b735-9a9e8ccc4324 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.612767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11b50a-e974-4733-a33d-be9c8ddd6bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.614163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11b50a-e974-4733-a33d-be9c8ddd6bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.614444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11b50a-e974-4733-a33d-be9c8ddd6bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.614795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11b50a-e974-4733-a33d-be9c8ddd6bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.615469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11b50a-e974-4733-a33d-be9c8ddd6bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.617404 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11b50a-e974-4733-a33d-be9c8ddd6bd3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.643670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 802/1581] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.644518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.655095 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23ed9-2536-4d6f-8afe-75a4202b1925 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.659599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.660790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.661077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.661351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.662022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.663952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.700943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 780/1582] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.700943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b915-900a-4642-a758-c60cb8db69b1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.700943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b915-900a-4642-a758-c60cb8db69b1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.700943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b915-900a-4642-a758-c60cb8db69b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.716409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5c96b915-900a-4642-a758-c60cb8db69b1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:08.717594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5c96b915-900a-4642-a758-c60cb8db69b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:08.725959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 781/1583] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:08.726694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b915-900a-4642-a758-c60cb8db69b1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.735556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b915-900a-4642-a758-c60cb8db69b1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.740766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.741409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.741686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.741954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.742658 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.744580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.752942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 803/1584] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.754193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.763489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ed333-e3f7-4cee-9da5-31aac9c73e7f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.768119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.768763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.769046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.769319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.770314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.771927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.782182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 804/1585] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.782926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.793620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9593d8-ea53-4c9f-a6d5-9b3447c76346 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.798540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.801469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.801469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.801469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.801469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.803563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.813980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 805/1586] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.814772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.824029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b99c8-6c1e-442d-9f02-740f036d5c2d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.826648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 782/1587] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:08 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:08.827532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.829307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.829957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.830253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.830539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.831238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.833589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.838081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4090641-93d9-4829-9ca2-b80d78be4d84 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.843999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 806/1588] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.844669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.846420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.847233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.848183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.848485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.850848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.850848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.855179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c33e7-ae9b-48fb-8550-ebb46f7d85c7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.860141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.860802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.861089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.861191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 783/1589] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.864797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.864797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.865019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.865114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.875667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 807/1590] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.875859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.885710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7fe3b-726e-414b-9d32-da60ba67a83a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.890533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.891180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.891465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.891775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.892490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.894127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.905070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 808/1591] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:08.905988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.916431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0876e0-a2d7-4a1d-b702-bdd61f12834f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.921197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.922109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.922223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed07843-c871-454c-a08b-cff5e8395273 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.922395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.922680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] PATH_INFO: `/v3/projects/458b23cc639f45899815590248afe298/users/7e4564c8f0b7416493f411749b4f9d9f/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.926521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.927455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.927883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.928188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.937476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.938987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=458b23cc639f45899815590248afe298, user_id=7e4564c8f0b7416493f411749b4f9d9f, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.939156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.940092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:15:08.940092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:15:08.940092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:15:08.942924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:08.950754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 784/1592] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:08.951851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:08.961843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec43bf6-db49-4e1e-9572-79741435ce33 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:08.966476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:08.967507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:08.967863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:08.968135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] PATH_INFO: `/v3/projects/d45888465c0f408a903d88d15bff69f2/users/8d4bbcec028b48e0ace0d2fe12ff5b49/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:08.982831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45888465c0f408a903d88d15bff69f2, user_id=8d4bbcec028b48e0ace0d2fe12ff5b49, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:08.985533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.015071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 809/1593] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:08 2022] PUT /identity/v3/projects/458b23cc639f45899815590248afe298/users/7e4564c8f0b7416493f411749b4f9d9f/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:09.016768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.075312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe4cbb-91d6-43b3-bfa7-958f8f350b44 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.080119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.087085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.087469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.087934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.088724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.090467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.100139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 785/1594] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:08 2022] PUT /identity/v3/projects/d45888465c0f408a903d88d15bff69f2/users/8d4bbcec028b48e0ace0d2fe12ff5b49/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:09.101495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.102024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 810/1595] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:09.103073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.122250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e690-5a95-4942-9801-ff432e2faa2f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.176993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a777bc-1557-4c7d-babf-1472f333b45f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.179915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.180615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.180894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.181173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.181776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.181920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.182552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.182897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.183229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.183464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.184117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.185760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.194062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 811/1596] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:09.194938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.196251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 786/1597] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:09.199899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.205031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16c11-b5a2-481d-b183-6a80e19b56c0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.209021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4c981-0eb5-44fb-9375-c592d90e899d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.209753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c28c8-a2a6-468a-9385-8bbef3faa520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.210402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c28c8-a2a6-468a-9385-8bbef3faa520 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.210679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c28c8-a2a6-468a-9385-8bbef3faa520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.210953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c28c8-a2a6-468a-9385-8bbef3faa520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.211685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c28c8-a2a6-468a-9385-8bbef3faa520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.213263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c28c8-a2a6-468a-9385-8bbef3faa520 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.213444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.214081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.214359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.214633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.215330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.216927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.223903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 812/1598] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:09.225193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305de639-66a3-42c1-8726-7796f063a810 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.225484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305de639-66a3-42c1-8726-7796f063a810 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.225755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305de639-66a3-42c1-8726-7796f063a810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.230595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 787/1599] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:09.230595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.239864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c37f8-f8d3-4210-83a8-e423fd7d8614 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.244524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a83196-6290-45a4-bf7b-0b5c39f66626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.245182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a83196-6290-45a4-bf7b-0b5c39f66626 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.245464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a83196-6290-45a4-bf7b-0b5c39f66626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.245736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a83196-6290-45a4-bf7b-0b5c39f66626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.246458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a83196-6290-45a4-bf7b-0b5c39f66626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.248103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a83196-6290-45a4-bf7b-0b5c39f66626 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.256885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-305de639-66a3-42c1-8726-7796f063a810 None None] MFA Rules not processed for user `7e4564c8f0b7416493f411749b4f9d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.258164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-305de639-66a3-42c1-8726-7796f063a810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.259888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 788/1600] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:09.261256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede3087-0de2-4226-ae76-8c035895b8db None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.261543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede3087-0de2-4226-ae76-8c035895b8db None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.261819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede3087-0de2-4226-ae76-8c035895b8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.289710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0ede3087-0de2-4226-ae76-8c035895b8db None None] MFA Rules not processed for user `8d4bbcec028b48e0ace0d2fe12ff5b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.291125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0ede3087-0de2-4226-ae76-8c035895b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.340694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 813/1601] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.341521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305de639-66a3-42c1-8726-7796f063a810 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.357570 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305de639-66a3-42c1-8726-7796f063a810 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.362931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.364772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.365060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.365342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] PATH_INFO: `/v3/projects/494afadf76c94f859b141309a6a3e37b/users/f3dc7683c04d42ffa5642c97a2f25d1d/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.367781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 789/1602] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.368960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede3087-0de2-4226-ae76-8c035895b8db None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.382100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494afadf76c94f859b141309a6a3e37b, user_id=f3dc7683c04d42ffa5642c97a2f25d1d, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.382855 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede3087-0de2-4226-ae76-8c035895b8db None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.385666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.387875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.388769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.389048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.389319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] PATH_INFO: `/v3/projects/56ba8d3ade6e4efd90ce4c340464c565/users/d38cdf40e33b47d8aa79e029cded7503/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.406961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ba8d3ade6e4efd90ce4c340464c565, user_id=d38cdf40e33b47d8aa79e029cded7503, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.409740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.452906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 814/1603] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:09 2022] PUT /identity/v3/projects/494afadf76c94f859b141309a6a3e37b/users/f3dc7683c04d42ffa5642c97a2f25d1d/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:09.453731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.476506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 790/1604] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:09 2022] PUT /identity/v3/projects/56ba8d3ade6e4efd90ce4c340464c565/users/d38cdf40e33b47d8aa79e029cded7503/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:09.477325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.517967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c053a7-23b9-481f-9ce5-cb0068f67fd6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.546169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5072f3d-643d-4c73-bdef-1149a29d2174 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.553015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.554020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.554369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.554707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] PATH_INFO: `/v3/projects/64990eb359b54b0a8636a6e7619800a7/users/7b559cc982a846bea7f7f2077b7c455b/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.570635 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64990eb359b54b0a8636a6e7619800a7, user_id=7b559cc982a846bea7f7f2077b7c455b, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.573353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.575417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436350a-9d03-466e-bac7-47e78bdf180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.576336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436350a-9d03-466e-bac7-47e78bdf180a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.576618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436350a-9d03-466e-bac7-47e78bdf180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.577019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436350a-9d03-466e-bac7-47e78bdf180a admin admin] PATH_INFO: `/v3/projects/dc83025212a2416ba02124f26c956a04/users/7ba85ecaa6d04e5aa2d6443ab0b19ba1/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.591468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436350a-9d03-466e-bac7-47e78bdf180a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc83025212a2416ba02124f26c956a04, user_id=7ba85ecaa6d04e5aa2d6443ab0b19ba1, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.594179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436350a-9d03-466e-bac7-47e78bdf180a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.634294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 791/1605] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:09 2022] PUT /identity/v3/projects/64990eb359b54b0a8636a6e7619800a7/users/7b559cc982a846bea7f7f2077b7c455b/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:09.637332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.656776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 815/1606] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:09 2022] PUT /identity/v3/projects/dc83025212a2416ba02124f26c956a04/users/7ba85ecaa6d04e5aa2d6443ab0b19ba1/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:09.669119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5f35-02c3-452d-b8c7-926980d0d0b2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.669407 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5f35-02c3-452d-b8c7-926980d0d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.669688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5f35-02c3-452d-b8c7-926980d0d0b2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.671223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 816/1607] 10.222.0.71 () {54 vars in 961 bytes} [Mon May 16 15:15:09 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:09.672238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6367-1a79-462b-bb8b-23bb3eb96dc8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.677637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6367-1a79-462b-bb8b-23bb3eb96dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.677637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6367-1a79-462b-bb8b-23bb3eb96dc8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.677637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 817/1608] 10.222.0.71 () {54 vars in 961 bytes} [Mon May 16 15:15:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:09.677637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f852a5-42f5-4908-bd3e-b3d84c633dc3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.677637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f852a5-42f5-4908-bd3e-b3d84c633dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.677637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f852a5-42f5-4908-bd3e-b3d84c633dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.698392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d1f852a5-42f5-4908-bd3e-b3d84c633dc3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.700709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d1f852a5-42f5-4908-bd3e-b3d84c633dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.707510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663f33-8cc9-4598-a7e8-62ce4daf2771 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:09.761039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 818/1609] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.764040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3722d2-4b5a-4a50-89f7-8c42756387f2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.764040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3722d2-4b5a-4a50-89f7-8c42756387f2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.764040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3722d2-4b5a-4a50-89f7-8c42756387f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.775241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff6f56-ca99-4128-9367-bffc27982507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:09.776296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff6f56-ca99-4128-9367-bffc27982507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.776484 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff6f56-ca99-4128-9367-bffc27982507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.776765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff6f56-ca99-4128-9367-bffc27982507 admin admin] PATH_INFO: `/v3/projects/0c695c22b383463eb79bebfa87026084/users/f5a810a177e940a7bf8689482305e7e8/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.782110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ed3722d2-4b5a-4a50-89f7-8c42756387f2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.783399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ed3722d2-4b5a-4a50-89f7-8c42756387f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.792472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 819/1610] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.795003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1e33-e873-40fc-98ff-3d2c8eded097 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.795003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1e33-e873-40fc-98ff-3d2c8eded097 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.795003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb1e33-e873-40fc-98ff-3d2c8eded097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.795316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff6f56-ca99-4128-9367-bffc27982507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c695c22b383463eb79bebfa87026084, user_id=f5a810a177e940a7bf8689482305e7e8, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:09.797721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff6f56-ca99-4128-9367-bffc27982507 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:09.818608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cdeb1e33-e873-40fc-98ff-3d2c8eded097 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.823716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cdeb1e33-e873-40fc-98ff-3d2c8eded097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.830803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 820/1611] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.832785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb08cf3-75a7-4443-a7d9-413cf01663bd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.833275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb08cf3-75a7-4443-a7d9-413cf01663bd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.833558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb08cf3-75a7-4443-a7d9-413cf01663bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.860287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7bb08cf3-75a7-4443-a7d9-413cf01663bd None None] MFA Rules not processed for user `7ba85ecaa6d04e5aa2d6443ab0b19ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.861482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7bb08cf3-75a7-4443-a7d9-413cf01663bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.872501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 792/1612] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:09 2022] PUT /identity/v3/projects/0c695c22b383463eb79bebfa87026084/users/f5a810a177e940a7bf8689482305e7e8/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:09.874037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17801f94-5b3a-44a0-b085-328d2f2a5644 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.874385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17801f94-5b3a-44a0-b085-328d2f2a5644 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.875094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17801f94-5b3a-44a0-b085-328d2f2a5644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.894017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-17801f94-5b3a-44a0-b085-328d2f2a5644 None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.896112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-17801f94-5b3a-44a0-b085-328d2f2a5644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.940179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 821/1613] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.941886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe41ad-1c49-421e-9be7-1fc16034d359 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:09.942595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe41ad-1c49-421e-9be7-1fc16034d359 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:09.943035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe41ad-1c49-421e-9be7-1fc16034d359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:09.963745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-13fe41ad-1c49-421e-9be7-1fc16034d359 None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:09.964972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-13fe41ad-1c49-421e-9be7-1fc16034d359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:09.966583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 793/1614] 10.222.0.71 () {58 vars in 1058 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.967888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17801f94-5b3a-44a0-b085-328d2f2a5644 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:09.976263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 822/1615] 10.222.0.71 () {58 vars in 1058 bytes} [Mon May 16 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:09.977182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe41ad-1c49-421e-9be7-1fc16034d359 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.035973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17801f94-5b3a-44a0-b085-328d2f2a5644 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.036927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe41ad-1c49-421e-9be7-1fc16034d359 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.041388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2c73-a21c-4f6f-bc15-c9ec767bbffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.042346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2c73-a21c-4f6f-bc15-c9ec767bbffe service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.042568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.042799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2c73-a21c-4f6f-bc15-c9ec767bbffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.043192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2c73-a21c-4f6f-bc15-c9ec767bbffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.043362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.043591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.043930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.105006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.106535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.108802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e2c73-a21c-4f6f-bc15-c9ec767bbffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.110456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e2c73-a21c-4f6f-bc15-c9ec767bbffe service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.117038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 823/1616] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:10.117976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.121159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 794/1617] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:10.122676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddc4dd-5157-4681-9bea-e1bd5d1acafe None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.122909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddc4dd-5157-4681-9bea-e1bd5d1acafe None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.123190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddc4dd-5157-4681-9bea-e1bd5d1acafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.126443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe2dc3-69ee-4b7f-904d-c34cfa9876c2 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.131275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.132209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.132676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.133154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.143090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.145322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.152323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-baddc4dd-5157-4681-9bea-e1bd5d1acafe None None] MFA Rules not processed for user `f5a810a177e940a7bf8689482305e7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:10.153531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-baddc4dd-5157-4681-9bea-e1bd5d1acafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:10.156552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 824/1618] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:10.157441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.171896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a6f5a-5cac-4f78-8f12-806a070f972e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.185514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.186202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.186526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.186801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.235151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 795/1619] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:10.236092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddc4dd-5157-4681-9bea-e1bd5d1acafe None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.248411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddc4dd-5157-4681-9bea-e1bd5d1acafe None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.249479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.251215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.255844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.256412 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.256703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.257032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.276446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 825/1620] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:10.277641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.314697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c96c-6b7d-44fa-bb7a-2860d8430d15 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.322663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c7ac5-6fa2-4b45-9abe-f2acba32a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.323402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c7ac5-6fa2-4b45-9abe-f2acba32a11e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.325987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c7ac5-6fa2-4b45-9abe-f2acba32a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.326366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c7ac5-6fa2-4b45-9abe-f2acba32a11e admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.327462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c7ac5-6fa2-4b45-9abe-f2acba32a11e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.331101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c7ac5-6fa2-4b45-9abe-f2acba32a11e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.359207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.361853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.389288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 796/1621] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:10.439317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 826/1622] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:10 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:10.444395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.462216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e29f4-f5e5-4bb7-b66a-ae921cbfec4d service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.535304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59e012-9e3b-49b8-a65b-b2923f9a9e73 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.535596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59e012-9e3b-49b8-a65b-b2923f9a9e73 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.535925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59e012-9e3b-49b8-a65b-b2923f9a9e73 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.537540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 827/1623] 10.222.0.71 () {54 vars in 912 bytes} [Mon May 16 15:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:10.544093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cff1a-ea11-4da6-a5eb-22f148dbc03a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.548131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cff1a-ea11-4da6-a5eb-22f148dbc03a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.548131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cff1a-ea11-4da6-a5eb-22f148dbc03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.580380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.581068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-110cff1a-ea11-4da6-a5eb-22f148dbc03a None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:10.581244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.581596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.582424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.583174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.583553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-110cff1a-ea11-4da6-a5eb-22f148dbc03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:10.585461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.678175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 828/1624] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:10.691544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cff1a-ea11-4da6-a5eb-22f148dbc03a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.763451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 797/1625] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:10 2022] POST /identity/v3/users => generated 300 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:10.765223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.795245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cff1a-ea11-4da6-a5eb-22f148dbc03a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.795660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056f929-4a32-4c4d-bf18-b0abbaee66d8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.804927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.805602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.805933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.806537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.808770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b3c48-4f2a-45bc-b198-3207cc197055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.809426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b3c48-4f2a-45bc-b198-3207cc197055 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.809828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b3c48-4f2a-45bc-b198-3207cc197055 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.810106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b3c48-4f2a-45bc-b198-3207cc197055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.825367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.827364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.845179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 829/1626] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:10.846563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.857822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99227a4-64d1-4b90-be1f-b6c915bef5b3 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.862677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.863507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.863890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.864226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.865020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.866681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.884424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 830/1627] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:10.888218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:10.901363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43972509-bd64-4ab0-bcf4-4680f19daaed admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:10.908379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:10.909466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:10.910415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:10.910758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] PATH_INFO: `/v3/projects/e5af1bb3fd5d4df08048d70a701806d5/users/60886317dff444a1ae5ffe7fad3da509/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:10.912567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b3c48-4f2a-45bc-b198-3207cc197055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.920583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b3c48-4f2a-45bc-b198-3207cc197055 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.933684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5af1bb3fd5d4df08048d70a701806d5, user_id=60886317dff444a1ae5ffe7fad3da509, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:10.940068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:10.943378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 798/1628] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:11.010761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c81c8c-61b3-4888-a1f2-cc9bc149f5e6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:11.011047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c81c8c-61b3-4888-a1f2-cc9bc149f5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:11.011324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c81c8c-61b3-4888-a1f2-cc9bc149f5e6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:11.014048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 799/1629] 10.222.0.71 () {54 vars in 912 bytes} [Mon May 16 15:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:11.020250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e816-1fac-454a-9ce9-bddbc2d89ae1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:11.020426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e816-1fac-454a-9ce9-bddbc2d89ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:11.020793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e816-1fac-454a-9ce9-bddbc2d89ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:11.046409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 831/1630] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:10 2022] PUT /identity/v3/projects/e5af1bb3fd5d4df08048d70a701806d5/users/60886317dff444a1ae5ffe7fad3da509/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:11.048132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:11.065274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-29f2e816-1fac-454a-9ce9-bddbc2d89ae1 None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:11.067099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-29f2e816-1fac-454a-9ce9-bddbc2d89ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:11.172047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc912baf-d743-413c-8025-c5e60373adbf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:11.181086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:11.181987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:11.182444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:11.182889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:11.184171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:11.185985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:11.201585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 832/1631] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:11.202720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:11.214505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce24649-718d-43a0-9e62-0fdbe9c8ddb4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:11.216192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 800/1632] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:11.227468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e816-1fac-454a-9ce9-bddbc2d89ae1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:11.229791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccc0cc-e628-46e9-8244-bb8760421e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:11.230883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccc0cc-e628-46e9-8244-bb8760421e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:11.231332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccc0cc-e628-46e9-8244-bb8760421e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:11.231787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccc0cc-e628-46e9-8244-bb8760421e6b admin admin] PATH_INFO: `/v3/projects/e5af1bb3fd5d4df08048d70a701806d5/users/60886317dff444a1ae5ffe7fad3da509/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:11.234114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccc0cc-e628-46e9-8244-bb8760421e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5af1bb3fd5d4df08048d70a701806d5, user_id=60886317dff444a1ae5ffe7fad3da509, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:11.237013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccc0cc-e628-46e9-8244-bb8760421e6b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:11.250974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 833/1633] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:11 2022] PUT /identity/v3/projects/e5af1bb3fd5d4df08048d70a701806d5/users/60886317dff444a1ae5ffe7fad3da509/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:11.263957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7ad4-9955-4467-98c1-603f1bd0ea3b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:11.264512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7ad4-9955-4467-98c1-603f1bd0ea3b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:11.264984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7ad4-9955-4467-98c1-603f1bd0ea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:11.322806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b04d7ad4-9955-4467-98c1-603f1bd0ea3b None None] MFA Rules not processed for user `60886317dff444a1ae5ffe7fad3da509`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:11.326239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b04d7ad4-9955-4467-98c1-603f1bd0ea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:11.358722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e816-1fac-454a-9ce9-bddbc2d89ae1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:11.433097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:11.435055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:11.435055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:11.435055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:11.451106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 834/1634] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:11.541191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:11.542650 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:11.587664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 801/1635] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:12.267920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d7ad4-9955-4467-98c1-603f1bd0ea3b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:12.280593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d7ad4-9955-4467-98c1-603f1bd0ea3b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:12.287584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e5208-48d2-4210-a4de-fcaff8d06f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:12.288709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e5208-48d2-4210-a4de-fcaff8d06f2b service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:12.289154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e5208-48d2-4210-a4de-fcaff8d06f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:12.289606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e5208-48d2-4210-a4de-fcaff8d06f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:12.301906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e5208-48d2-4210-a4de-fcaff8d06f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:12.303797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e5208-48d2-4210-a4de-fcaff8d06f2b service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:12.318224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 835/1636] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:12.371480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:12.469152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418abd07-c0e9-4e46-b887-246db87942d3 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:12.475189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:12.476279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:12.476752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:12.477145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] PATH_INFO: `/v3/users/f5a810a177e940a7bf8689482305e7e8` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:12.478458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a810a177e940a7bf8689482305e7e8)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:12.480529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:12.737747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d799-c13b-4aee-865a-de0d5fe27204 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:12.738072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d799-c13b-4aee-865a-de0d5fe27204 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:12.738347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d799-c13b-4aee-865a-de0d5fe27204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:12.822641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a810a177e940a7bf8689482305e7e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:12.827860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5f75d799-c13b-4aee-865a-de0d5fe27204 None None] MFA Rules not processed for user `f3dc7683c04d42ffa5642c97a2f25d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:12.833950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5f75d799-c13b-4aee-865a-de0d5fe27204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:12.976317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a810a177e940a7bf8689482305e7e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:12.979313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a810a177e940a7bf8689482305e7e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.005706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a810a177e940a7bf8689482305e7e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.017242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a810a177e940a7bf8689482305e7e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.042090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 802/1637] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:12 2022] DELETE /identity/v3/users/f5a810a177e940a7bf8689482305e7e8 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:13.045785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:13.049778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:13.095518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 836/1638] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:13.097052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844379-10a5-4efc-8c2f-3f7e0f83a776 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:13.099455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844379-10a5-4efc-8c2f-3f7e0f83a776 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:13.099768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844379-10a5-4efc-8c2f-3f7e0f83a776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:13.160608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1a844379-10a5-4efc-8c2f-3f7e0f83a776 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:13.161977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a844379-10a5-4efc-8c2f-3f7e0f83a776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:13.215054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba224d-8489-4307-a527-24920496b1f4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:13.218640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:13.220021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:13.220512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:13.220900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] PATH_INFO: `/v3/users/8d4bbcec028b48e0ace0d2fe12ff5b49` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:13.222907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4bbcec028b48e0ace0d2fe12ff5b49)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:13.225775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:13.283192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 837/1639] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:13.284731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:13.285129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:13.285557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:13.318208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] MFA Rules not processed for user `f3dc7683c04d42ffa5642c97a2f25d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:13.320013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:13.408613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 838/1640] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:13.409807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:13.412150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:13.480245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4bbcec028b48e0ace0d2fe12ff5b49'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.507958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3aab6-b7de-4e65-b64b-7bd73a0fcb87 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:13.512769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:13.513576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:13.513908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:13.514188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:13.536046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:13.542514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:13.570776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4bbcec028b48e0ace0d2fe12ff5b49'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.571211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 839/1641] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:13.571694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4bbcec028b48e0ace0d2fe12ff5b49'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.572522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:13.577788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:13.591954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4bbcec028b48e0ace0d2fe12ff5b49'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.609059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-03d9e777-bfe6-4e12-a43c-2bc192db2f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4bbcec028b48e0ace0d2fe12ff5b49'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:13.620414 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 803/1642] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:13 2022] DELETE /identity/v3/users/8d4bbcec028b48e0ace0d2fe12ff5b49 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:13.625074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:13.625707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:13.626177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:13.668340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:13.671035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:13.746704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eece6d-bb09-4065-b448-82fbce4c3522 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:13.812060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 804/1643] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:13.833016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:13.834493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:13.901109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdb8b0-f5b0-4ec2-80e6-42f49a25692e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:13.901109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdb8b0-f5b0-4ec2-80e6-42f49a25692e service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:13.901322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdb8b0-f5b0-4ec2-80e6-42f49a25692e service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:13.901515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdb8b0-f5b0-4ec2-80e6-42f49a25692e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:14.210997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-42cdb8b0-f5b0-4ec2-80e6-42f49a25692e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:14.219455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f511e-58f6-4128-a766-1e36b48aa1a4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:14.219455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126e406-f457-412f-a75f-b113712f1c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:14.219455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e406-f457-412f-a75f-b113712f1c89 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:14.219455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e406-f457-412f-a75f-b113712f1c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:14.220613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e406-f457-412f-a75f-b113712f1c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:14.220613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126e406-f457-412f-a75f-b113712f1c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:14.220613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126e406-f457-412f-a75f-b113712f1c89 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:14.220613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 805/1644] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:14.225847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdb8b0-f5b0-4ec2-80e6-42f49a25692e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:14.225847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdb8b0-f5b0-4ec2-80e6-42f49a25692e service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:14.225847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 840/1645] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:15.048586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec2091-f6c1-4bfc-9664-e9ce02b0d23d None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:15.049473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec2091-f6c1-4bfc-9664-e9ce02b0d23d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:15.049946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec2091-f6c1-4bfc-9664-e9ce02b0d23d None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:15.052639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 806/1646] 10.222.0.71 () {54 vars in 912 bytes} [Mon May 16 15:15:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:15.059289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:15.059770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:15.059947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:15.105633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] MFA Rules not processed for user `76b2a2d381a14f5fb04e721905b36ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:15.109587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:15.132217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 841/1647] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:15:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:15.148526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec2091-f6c1-4bfc-9664-e9ce02b0d23d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:15.168160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec2091-f6c1-4bfc-9664-e9ce02b0d23d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:15.179232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:15.179982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:15.180259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:15.181496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:15.200606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:15.203052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:15.233438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 807/1648] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:15.540215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:15.543568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:15.586304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d412a1-8ec9-4671-b22d-465b5c8deaed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:15.599989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:15.601752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:15.603252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:15.604012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] PATH_INFO: `/v3/projects/0c695c22b383463eb79bebfa87026084` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:15.609130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c695c22b383463eb79bebfa87026084)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:15.611315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:15.724697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:15.732283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:15.785441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6811d43-a601-40c8-9510-a3c5215fa52a service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:15.815746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c695c22b383463eb79bebfa87026084', 'request_id': 'req-638b219a-efb7-41f7-9c50-1537cda090f7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:15.837905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c695c22b383463eb79bebfa87026084', 'request_id': 'req-638b219a-efb7-41f7-9c50-1537cda090f7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:15.859718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c695c22b383463eb79bebfa87026084', 'request_id': 'req-638b219a-efb7-41f7-9c50-1537cda090f7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:15.942469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:15.945035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:15.945317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:15.947727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] PATH_INFO: `/v3/users/7e4564c8f0b7416493f411749b4f9d9f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:15.955381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4564c8f0b7416493f411749b4f9d9f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:15.958081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:15.981309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] The token cache is being invalidate because project 0c695c22b383463eb79bebfa87026084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:15.981931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:15.982772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:15.989234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 842/1649] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:15 2022] DELETE /identity/v3/projects/0c695c22b383463eb79bebfa87026084 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:15.995032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:15.996849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:16.038846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b219a-efb7-41f7-9c50-1537cda090f7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:16.049443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:16.051228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:16.051783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:16.052777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] PATH_INFO: `/v3/users/60886317dff444a1ae5ffe7fad3da509` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:16.056463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60886317dff444a1ae5ffe7fad3da509)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:16.060304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:15:16.060304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 16 15:15:16.060304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: May 16 15:15:16.063882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:16.255264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4564c8f0b7416493f411749b4f9d9f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.396301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4564c8f0b7416493f411749b4f9d9f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.397704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4564c8f0b7416493f411749b4f9d9f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.402045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60886317dff444a1ae5ffe7fad3da509'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.437233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4564c8f0b7416493f411749b4f9d9f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.452285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4564c8f0b7416493f411749b4f9d9f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.481315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 808/1650] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:15 2022] DELETE /identity/v3/users/7e4564c8f0b7416493f411749b4f9d9f => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:16.484312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:16.487610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60886317dff444a1ae5ffe7fad3da509'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.491501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:16.516129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60886317dff444a1ae5ffe7fad3da509'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.516994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60886317dff444a1ae5ffe7fad3da509'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.598295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-baf65dc1-0d46-46f5-92de-402dbb8968a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60886317dff444a1ae5ffe7fad3da509'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:16.619800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 843/1651] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:15 2022] DELETE /identity/v3/users/60886317dff444a1ae5ffe7fad3da509 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:16.621878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219883f5-29c4-44f8-bd77-8c4b10271700 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:16.630582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219883f5-29c4-44f8-bd77-8c4b10271700 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:16.630880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219883f5-29c4-44f8-bd77-8c4b10271700 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:16.649631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 844/1652] 10.222.0.71 () {54 vars in 960 bytes} [Mon May 16 15:15:16 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:16.652134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b466e99-c9c5-429f-ba44-690e6e3384b2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:16.652610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b466e99-c9c5-429f-ba44-690e6e3384b2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:16.653021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b466e99-c9c5-429f-ba44-690e6e3384b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:16.657038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042e3f1-0f29-49fc-9919-814c9f6b667a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:16.691934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4b466e99-c9c5-429f-ba44-690e6e3384b2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:16.693379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4b466e99-c9c5-429f-ba44-690e6e3384b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:16.751521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:16.755304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:16.755689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:16.756045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] PATH_INFO: `/v3/projects/d45888465c0f408a903d88d15bff69f2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:16.758103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45888465c0f408a903d88d15bff69f2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:16.759407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 15:15:16.759407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 16 15:15:16.759407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: May 16 15:15:16.760861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:16.794984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 845/1653] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:16.796694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:16.797125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:16.797608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:16.849724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] MFA Rules not processed for user `6c36906c508f41cca018e5f6dcaadd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:16.850514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:16.987221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45888465c0f408a903d88d15bff69f2', 'request_id': 'req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:17.051584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 846/1654] 10.222.0.71 () {58 vars in 1057 bytes} [Mon May 16 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:17.053436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:17.054573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:17.064559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45888465c0f408a903d88d15bff69f2', 'request_id': 'req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:17.074205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45888465c0f408a903d88d15bff69f2', 'request_id': 'req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:17.126107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] The token cache is being invalidate because project d45888465c0f408a903d88d15bff69f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:17.126593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:17.130569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:17.135310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 809/1655] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:16 2022] DELETE /identity/v3/projects/d45888465c0f408a903d88d15bff69f2 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:17.138646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:17.142804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:17.198448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec635f0-18dd-4324-bd84-8f8b1fa42a0a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:17.200077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cea246-f293-496c-b0a9-c2926946d86b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:17.210440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:17.210759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac401a00-3fa4-407d-8c81-7d7f601f47a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:17.211479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac401a00-3fa4-407d-8c81-7d7f601f47a9 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:17.211805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac401a00-3fa4-407d-8c81-7d7f601f47a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:17.212068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac401a00-3fa4-407d-8c81-7d7f601f47a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:17.213630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:17.214042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:17.215066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:17.216048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ac401a00-3fa4-407d-8c81-7d7f601f47a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:17.218747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:17.230194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac401a00-3fa4-407d-8c81-7d7f601f47a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:17.232156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac401a00-3fa4-407d-8c81-7d7f601f47a9 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:17.245052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 810/1656] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:17.246703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea820e1-7f6c-4c42-8c06-d0b7fa93dde2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:17.247165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea820e1-7f6c-4c42-8c06-d0b7fa93dde2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:17.247667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea820e1-7f6c-4c42-8c06-d0b7fa93dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:17.284130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6ea820e1-7f6c-4c42-8c06-d0b7fa93dde2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:17.286242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6ea820e1-7f6c-4c42-8c06-d0b7fa93dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:17.376909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:17.383733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:17.402168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 811/1657] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:17.404852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04eeeb9-46f3-491c-a00c-a2d53ff61d1e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:17.405361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04eeeb9-46f3-491c-a00c-a2d53ff61d1e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:17.405848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04eeeb9-46f3-491c-a00c-a2d53ff61d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:17.407444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 847/1658] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:17.446931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c04eeeb9-46f3-491c-a00c-a2d53ff61d1e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:17.448283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c04eeeb9-46f3-491c-a00c-a2d53ff61d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:17.470092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 812/1659] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:17.475001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:17.501173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b989f1-4f84-42f0-a5ad-1394b75cb47c service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:17.516352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:17.517443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:17.518482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:17.518761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:17.519548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:17.519904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:17.532803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:17.569100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 848/1660] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:17.569885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04eeeb9-46f3-491c-a00c-a2d53ff61d1e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:17.579180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04eeeb9-46f3-491c-a00c-a2d53ff61d1e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:17.608567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780751c-7de6-4621-85a0-1ec8af254eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:17.609268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780751c-7de6-4621-85a0-1ec8af254eea admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:17.609551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780751c-7de6-4621-85a0-1ec8af254eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:17.609822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780751c-7de6-4621-85a0-1ec8af254eea admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:17.610528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780751c-7de6-4621-85a0-1ec8af254eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:17.629706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780751c-7de6-4621-85a0-1ec8af254eea admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:17.767373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 813/1661] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:17 2022] POST /identity/v3/projects => generated 403 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:17.773289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:17.794569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aeaaad-067e-4938-9b60-36c65671814d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:17.975795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:17.976812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:17.977543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:17.979393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:17.980076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:17.985361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:18.099565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7455-f6e2-490c-9856-6577ad281acf None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.099878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7455-f6e2-490c-9856-6577ad281acf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.100152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f7455-f6e2-490c-9856-6577ad281acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.110291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 849/1662] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:17 2022] POST /identity/v3/users => generated 302 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:18.120282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:18.161851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284ca53-d6b6-46f2-b21d-c8bcee7c1313 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:18.185826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:18.186516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.186808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.187069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.187818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:18.189462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:18.193105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-fe5f7455-f6e2-490c-9856-6577ad281acf None None] MFA Rules not processed for user `97021ddaffff4e088961ba1873a8e697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:18.194337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fe5f7455-f6e2-490c-9856-6577ad281acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:18.223929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 850/1663] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:18.226262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:18.245159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e92ed1-c574-4349-84e1-bc31c22361cb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:18.254632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:18.256201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.256642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.258614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.258614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:18.261361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:18.282557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 851/1664] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:18.287448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:18.303491 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65b7c4-b551-44f1-bd03-60ce9a93d116 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:18.310880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad28fc-4b01-4e38-9837-1467f01513a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:18.311870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad28fc-4b01-4e38-9837-1467f01513a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.316726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad28fc-4b01-4e38-9837-1467f01513a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.316726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad28fc-4b01-4e38-9837-1467f01513a4 admin admin] PATH_INFO: `/v3/projects/3f804032882d4ed7ba65b5969dd00ea8/users/2a614c90163d425a9b04ed8574bb2a91/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.341915 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 814/1665] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:18.345739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.346027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.348091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.349311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad28fc-4b01-4e38-9837-1467f01513a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f804032882d4ed7ba65b5969dd00ea8, user_id=2a614c90163d425a9b04ed8574bb2a91, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:18.361132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad28fc-4b01-4e38-9837-1467f01513a4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:18.441530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] MFA Rules not processed for user `97021ddaffff4e088961ba1873a8e697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:18.446071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:18.474033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 852/1666] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:18 2022] PUT /identity/v3/projects/3f804032882d4ed7ba65b5969dd00ea8/users/2a614c90163d425a9b04ed8574bb2a91/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:18.476316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.476871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.477300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.482356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 815/1667] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:18.483476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:18.490589 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:18.508463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:18.510167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:18.645901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 853/1668] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:18.673576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:18.673576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:18.701402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2b86c-00e3-4bcc-a981-a1e3828c53e2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:18.796665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d413da-c367-4e8c-a4f5-25a8c5534ac2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:18.804023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:18.804925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.805348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.805756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.825525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:18.827388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:18.859749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:18.860710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:18.861062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:18.861412 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:18.862287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 854/1669] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:15:18.996881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:18.999911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:19.015309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 816/1670] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:19.161087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:19.168284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:19.194355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a2d1-bbf5-498e-a07f-6c112cb86bbe service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:19.208374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:19.210218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:19.210218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:19.210218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] PATH_INFO: `/v3/projects/e5af1bb3fd5d4df08048d70a701806d5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:19.215061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5af1bb3fd5d4df08048d70a701806d5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:19.217202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:19.435073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:19.436557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:19.478081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a389b-1e05-440b-bdb8-fc8135fb54d9 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:19.485599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5af1bb3fd5d4df08048d70a701806d5', 'request_id': 'req-0702d7b5-964b-4619-949c-1476a9b1bc11'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:19.517017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5af1bb3fd5d4df08048d70a701806d5', 'request_id': 'req-0702d7b5-964b-4619-949c-1476a9b1bc11'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:19.532482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5af1bb3fd5d4df08048d70a701806d5', 'request_id': 'req-0702d7b5-964b-4619-949c-1476a9b1bc11'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:19.632390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:19.633453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:19.634023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:19.634435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] PATH_INFO: `/v3/projects/458b23cc639f45899815590248afe298` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:19.638240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=458b23cc639f45899815590248afe298)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:19.640827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:19.658688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] The token cache is being invalidate because project e5af1bb3fd5d4df08048d70a701806d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:19.658985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:19.661036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0702d7b5-964b-4619-949c-1476a9b1bc11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:19.663928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 855/1671] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:19 2022] DELETE /identity/v3/projects/e5af1bb3fd5d4df08048d70a701806d5 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:19.670947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032164b-4d78-4a55-8193-d5cb35981ab6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:19.671236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032164b-4d78-4a55-8193-d5cb35981ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:19.671512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032164b-4d78-4a55-8193-d5cb35981ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:19.770775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9032164b-4d78-4a55-8193-d5cb35981ab6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:19.774023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9032164b-4d78-4a55-8193-d5cb35981ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:19.793648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 856/1672] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:19.797163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa923d7-f150-4b94-b7ec-85242b701fc1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:19.797614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa923d7-f150-4b94-b7ec-85242b701fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:19.798030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa923d7-f150-4b94-b7ec-85242b701fc1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:19.808232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 857/1673] 10.222.0.71 () {54 vars in 912 bytes} [Mon May 16 15:15:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:19.810059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cff102-1d97-4605-bb5d-5fb300a328a7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:19.811733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cff102-1d97-4605-bb5d-5fb300a328a7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:19.812159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cff102-1d97-4605-bb5d-5fb300a328a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:19.836002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-73cff102-1d97-4605-bb5d-5fb300a328a7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:19.837547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73cff102-1d97-4605-bb5d-5fb300a328a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:19.838970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '458b23cc639f45899815590248afe298', 'request_id': 'req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:19.924178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 858/1674] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:19.924178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b91460-e9b5-47f7-b555-9ceca4514d8c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:19.924178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b91460-e9b5-47f7-b555-9ceca4514d8c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:19.924178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b91460-e9b5-47f7-b555-9ceca4514d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:19.972668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-63b91460-e9b5-47f7-b555-9ceca4514d8c None None] MFA Rules not processed for user `76b2a2d381a14f5fb04e721905b36ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:19.976318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-63b91460-e9b5-47f7-b555-9ceca4514d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:19.985177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '458b23cc639f45899815590248afe298', 'request_id': 'req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:19.994133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '458b23cc639f45899815590248afe298', 'request_id': 'req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:20.020667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] The token cache is being invalidate because project 458b23cc639f45899815590248afe298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:20.021344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:20.022456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:20.026740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 817/1675] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:19 2022] DELETE /identity/v3/projects/458b23cc639f45899815590248afe298 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:20.027096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.032993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:20.063798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53d012-1ed5-4bbb-8da5-e81ded2740ba admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.070679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.071372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.071689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.071974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.073182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:20.073574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.075256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.090035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 859/1676] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:15:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:20.092044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09521cca-7ea6-49a1-bb03-b567f9928297 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.092405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09521cca-7ea6-49a1-bb03-b567f9928297 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.092754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09521cca-7ea6-49a1-bb03-b567f9928297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.117819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-09521cca-7ea6-49a1-bb03-b567f9928297 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:20.119316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09521cca-7ea6-49a1-bb03-b567f9928297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:20.120752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 818/1677] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:20.123238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.131003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 860/1678] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:20.132639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09521cca-7ea6-49a1-bb03-b567f9928297 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.133562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:20.163818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09521cca-7ea6-49a1-bb03-b567f9928297 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.174422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.175111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.175431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.176016 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.177275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.178077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55296d49-aa48-490d-9ec8-685e851d1887 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.179412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.198679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7a7c1-cc35-45c5-8e82-3147501505af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.199610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7a7c1-cc35-45c5-8e82-3147501505af service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.200086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7a7c1-cc35-45c5-8e82-3147501505af service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.200507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7a7c1-cc35-45c5-8e82-3147501505af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.218267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7a7c1-cc35-45c5-8e82-3147501505af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.220816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7a7c1-cc35-45c5-8e82-3147501505af service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.242209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 819/1679] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:20.245191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845182b2-e53f-43e6-9b50-a79b5f539ac2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.245564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845182b2-e53f-43e6-9b50-a79b5f539ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.247126 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845182b2-e53f-43e6-9b50-a79b5f539ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.267252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 861/1680] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:20 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:20.275101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.289489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-845182b2-e53f-43e6-9b50-a79b5f539ac2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:20.290724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-845182b2-e53f-43e6-9b50-a79b5f539ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:20.415754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bac931-6e97-4a86-8325-551ba9978372 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.443964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 820/1681] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:20.444449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.444938 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845182b2-e53f-43e6-9b50-a79b5f539ac2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.445482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.445782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.446110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.446832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.449879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.466702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845182b2-e53f-43e6-9b50-a79b5f539ac2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.477801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.478741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.479312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.479796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.480780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:20.481243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.483029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.510372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 821/1682] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:20.515531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.536422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81243332-886b-4814-b64a-5408aa50b8cb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.553566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.554699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.555120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.555528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.556438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.558957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.601707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 862/1683] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:20 2022] POST /identity/v3/users => generated 285 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:20.605957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.628178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d496f-7ef2-48b5-9abc-ba671f949b9d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.633044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.633726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.634010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.636060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.637797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.642139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.669407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 822/1684] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:20 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:20.669581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 863/1685] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:20.669659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.672212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.708343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec23dd4-12c4-41e4-bfce-da34a6502efb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.828633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56991-35c0-4fbf-a3be-97ecb0079508 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.856073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.856073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.856073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.856073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.860047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.864134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.887233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.888320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.892096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 823/1686] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:20.892825 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.893313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.894933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.896182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:20.897888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:20.917361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ac7ea-be17-4845-88f1-b59e6cddf510 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:20.923284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69113076-e2c5-43db-8d9f-0a8cd2959b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:20.924534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113076-e2c5-43db-8d9f-0a8cd2959b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:20.925218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113076-e2c5-43db-8d9f-0a8cd2959b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:20.925749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113076-e2c5-43db-8d9f-0a8cd2959b68 admin admin] PATH_INFO: `/v3/projects/ba742161ca6d4c4393f0aa1213c5d4ea/users/9aa1d320b70d40cc9286558451f3efa2/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:20.944884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69113076-e2c5-43db-8d9f-0a8cd2959b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba742161ca6d4c4393f0aa1213c5d4ea, user_id=9aa1d320b70d40cc9286558451f3efa2, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:20.957959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69113076-e2c5-43db-8d9f-0a8cd2959b68 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:21.035341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 864/1687] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:20 2022] POST /identity/v3/users => generated 290 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:21.039921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:21.059056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03282a-3290-4792-89c0-e0e5b18544da admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:21.070610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:21.071261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.072000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.072364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.073959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:21.075141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:21.080752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 824/1688] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:20 2022] PUT /identity/v3/projects/ba742161ca6d4c4393f0aa1213c5d4ea/users/9aa1d320b70d40cc9286558451f3efa2/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:21.092083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 865/1689] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:21.092504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.092864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.093213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:21.093616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.104645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccace8aa-8424-47ed-8902-a8aeb60873a2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:21.163540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] MFA Rules not processed for user `9aa1d320b70d40cc9286558451f3efa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:21.165388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:21.229445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:21.230488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.230969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.231441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.232405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:21.238417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:21.259936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 866/1690] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:21.267067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:21.291422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839a65-79ef-4645-b1d2-18a1d3509d40 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:21.292499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 825/1691] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:21.307508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:21.310562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:21.310902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a399c05-07f5-432c-80f4-2b66511c988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:21.311863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a399c05-07f5-432c-80f4-2b66511c988a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.312150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a399c05-07f5-432c-80f4-2b66511c988a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.312432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a399c05-07f5-432c-80f4-2b66511c988a admin admin] PATH_INFO: `/v3/projects/57148ac591604d0f8e2bf158675f5f11/users/cf50192ebb4f4de29739b6485674129c/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.344536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a399c05-07f5-432c-80f4-2b66511c988a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57148ac591604d0f8e2bf158675f5f11, user_id=cf50192ebb4f4de29739b6485674129c, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:21.347837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a399c05-07f5-432c-80f4-2b66511c988a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:21.426178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b7a8d-3cf7-4dd8-b384-f22039a83dd4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:21.430793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:21.431813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.432435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.432942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.445366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:21.448340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:21.453060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 867/1692] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:21 2022] PUT /identity/v3/projects/57148ac591604d0f8e2bf158675f5f11/users/cf50192ebb4f4de29739b6485674129c/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:21.465053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1edf8a-a176-486b-8634-2df9d4106dbf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.465713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1edf8a-a176-486b-8634-2df9d4106dbf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.466137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1edf8a-a176-486b-8634-2df9d4106dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.470981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 826/1693] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:15:21.495998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:21.510215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2376af-bc0c-4c8c-9631-f00b7af34d8c service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:21.536161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ec1edf8a-a176-486b-8634-2df9d4106dbf None None] MFA Rules not processed for user `cf50192ebb4f4de29739b6485674129c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:21.538321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ec1edf8a-a176-486b-8634-2df9d4106dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:21.674865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:21.680535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.680801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.681080 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] PATH_INFO: `/v3/users/9aa1d320b70d40cc9286558451f3efa2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.684791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 868/1694] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:21.685678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa1d320b70d40cc9286558451f3efa2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:21.685806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1edf8a-a176-486b-8634-2df9d4106dbf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:21.689570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:21.719698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1edf8a-a176-486b-8634-2df9d4106dbf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:21.733407 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c02b-34a0-43c2-96e3-f84ee2d376cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:21.734919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c02b-34a0-43c2-96e3-f84ee2d376cc admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.736457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c02b-34a0-43c2-96e3-f84ee2d376cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.736747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c02b-34a0-43c2-96e3-f84ee2d376cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.740920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c02b-34a0-43c2-96e3-f84ee2d376cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:21.744896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c02b-34a0-43c2-96e3-f84ee2d376cc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:21.952417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 869/1695] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:21 2022] POST /identity/v3/projects => generated 379 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:21.956409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554559f8-3df6-4767-924a-20022185975c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:21.956749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554559f8-3df6-4767-924a-20022185975c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:21.957142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554559f8-3df6-4767-924a-20022185975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:21.976626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa1d320b70d40cc9286558451f3efa2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:22.037745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-554559f8-3df6-4767-924a-20022185975c None None] MFA Rules not processed for user `97021ddaffff4e088961ba1873a8e697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:22.038999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-554559f8-3df6-4767-924a-20022185975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:22.063838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa1d320b70d40cc9286558451f3efa2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:22.064807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa1d320b70d40cc9286558451f3efa2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:22.120547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa1d320b70d40cc9286558451f3efa2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:22.146113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e9ed0469-41cb-44cb-b3ab-883d13283676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa1d320b70d40cc9286558451f3efa2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:22.164481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 827/1696] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:21 2022] DELETE /identity/v3/users/9aa1d320b70d40cc9286558451f3efa2 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:22.166112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2bc58-0771-49d0-bad7-d2b9ec8e30ab None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:22.167210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2bc58-0771-49d0-bad7-d2b9ec8e30ab None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:22.167722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2bc58-0771-49d0-bad7-d2b9ec8e30ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:22.211866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 870/1697] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:22.218182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554559f8-3df6-4767-924a-20022185975c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:22.218503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-61d2bc58-0771-49d0-bad7-d2b9ec8e30ab None None] MFA Rules not processed for user `2a614c90163d425a9b04ed8574bb2a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:22.218503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-61d2bc58-0771-49d0-bad7-d2b9ec8e30ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:22.222939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-554559f8-3df6-4767-924a-20022185975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:22.395502 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554559f8-3df6-4767-924a-20022185975c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:22.396929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 828/1698] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:22.398679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:22.401276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:22.401730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:22.443017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:22.444826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:22.525153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:22.525284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:22.525979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:22.525979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:22.526480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:22.530681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:22.586661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 829/1699] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:22.588182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:22.589562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:22.625040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf16b2-574e-4809-b11c-84f175ade451 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:22.641118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:22.643103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:22.647403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:22.648760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] PATH_INFO: `/v3/users/7ba85ecaa6d04e5aa2d6443ab0b19ba1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:22.651433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba85ecaa6d04e5aa2d6443ab0b19ba1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:22.658365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 871/1700] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:22 2022] POST /identity/v3/users => generated 296 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:22.659465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:22.662379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:22.662911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:22.697445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f63682-41b3-43fb-b960-0d4c3e05898b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:22.710247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:22.711141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:22.711544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:22.711955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:22.712771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:22.714852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:22.848478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 872/1701] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:22 2022] POST /identity/v3/projects => generated 405 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:22.858058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:22.995379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58fffe-3538-4180-96ac-2c38f8375da3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:23.002258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:23.003069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:23.003475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:23.003987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:23.004960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:23.009301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:23.021713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba85ecaa6d04e5aa2d6443ab0b19ba1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:23.039844 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 873/1702] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:23.041057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:23.048023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:23.108401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba85ecaa6d04e5aa2d6443ab0b19ba1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:23.109068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba85ecaa6d04e5aa2d6443ab0b19ba1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:23.154232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba85ecaa6d04e5aa2d6443ab0b19ba1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:23.195217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7e21747b-e0f6-4105-8ad3-65db9416cc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba85ecaa6d04e5aa2d6443ab0b19ba1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:23.209724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 830/1703] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:22 2022] DELETE /identity/v3/users/7ba85ecaa6d04e5aa2d6443ab0b19ba1 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:23.211135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:23.211427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:23.211739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:23.227198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581bade-9b04-4591-8369-655d7a726aae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:23.260092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:23.261257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:23.370362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:23.371386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:23.372664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:23.372852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:23.376676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:23.415474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 831/1704] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:23.416590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:23.418694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:23.560745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:23.569150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:23.603858 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304c956-9b29-4e14-a891-635982a72cc8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:23.606975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 874/1705] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:23.608399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:23.616521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:23.630510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe017f9e-aa23-49c2-8211-c39f0e905a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:23.631333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe017f9e-aa23-49c2-8211-c39f0e905a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:23.631654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe017f9e-aa23-49c2-8211-c39f0e905a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:23.631932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe017f9e-aa23-49c2-8211-c39f0e905a2d admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:23.632646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe017f9e-aa23-49c2-8211-c39f0e905a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:23.634619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe017f9e-aa23-49c2-8211-c39f0e905a2d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:23.652991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdd07f-3361-49e1-a371-49477d3891dd service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:23.666440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:23.667827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:23.668321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:23.668798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] PATH_INFO: `/v3/projects/9d74110e80ba42b896f99b2234bb1ad3/users/ac5c9e1e6ce348e39871f20f835028e1/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:23.689238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d74110e80ba42b896f99b2234bb1ad3, user_id=ac5c9e1e6ce348e39871f20f835028e1, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:23.694980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:23.750776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 832/1706] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:23 2022] POST /identity/v3/users => generated 309 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:23.752431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e706c-f593-44ca-9e0c-60774c515fbd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:23.752725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e706c-f593-44ca-9e0c-60774c515fbd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:23.753007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e706c-f593-44ca-9e0c-60774c515fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:23.791816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c78e706c-f593-44ca-9e0c-60774c515fbd None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:23.796705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c78e706c-f593-44ca-9e0c-60774c515fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:23.817069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 833/1707] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:23.822934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e706c-f593-44ca-9e0c-60774c515fbd None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:23.823015 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 875/1708] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:23 2022] PUT /identity/v3/projects/9d74110e80ba42b896f99b2234bb1ad3/users/ac5c9e1e6ce348e39871f20f835028e1/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:23.823015 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:23.858319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e706c-f593-44ca-9e0c-60774c515fbd None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:23.975740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:23.976671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:23.977101 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:23.977601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:23.979006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:23.981061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:23.992908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb76a-fbef-4a65-bd72-54f1b637372a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:24.000871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 834/1709] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:24.001388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:24.002306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.002756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.003149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:24.005009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:24.010395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:24.010530 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:24.010615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:24.021282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 876/1710] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:24.022680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:24.025574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:24.229823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0cbf5-cc23-4a78-9b2e-138eebe2eb0a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:24.244364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:24.245920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.245920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.246374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:24.253317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a0ef4-c1de-4c0a-9bbb-dbd0b5167e57 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:24.261657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:24.262592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.263128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.263553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:24.298105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:24.307508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:24.343724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 835/1711] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:24.350848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:24.378050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dd9e3-0868-4ec2-8d95-04c72607bd9a service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:24.391403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:24.392571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.392930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.393281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] PATH_INFO: `/v3/projects/2aaa2084ed904b08bacb9cc5679a40fb/users/81787c270dce4455831c24d106aaa92f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:24.457905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:24.459654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:24.461593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aaa2084ed904b08bacb9cc5679a40fb, user_id=81787c270dce4455831c24d106aaa92f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:24.469851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:24.486886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 877/1712] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:15:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:15:24.487821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:24.524950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a861da-5d83-4e5c-b87c-fc4d95f20a2c service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:24.563291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde5098-906b-48e1-aa5f-f1b9ffbc3e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:24.563847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde5098-906b-48e1-aa5f-f1b9ffbc3e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.564133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde5098-906b-48e1-aa5f-f1b9ffbc3e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.564413 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde5098-906b-48e1-aa5f-f1b9ffbc3e6c admin admin] PATH_INFO: `/v3/projects/9d74110e80ba42b896f99b2234bb1ad3/users/ac5c9e1e6ce348e39871f20f835028e1/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:24.565252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 836/1713] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:24 2022] PUT /identity/v3/projects/2aaa2084ed904b08bacb9cc5679a40fb/users/81787c270dce4455831c24d106aaa92f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:24.568936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:24.573330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde5098-906b-48e1-aa5f-f1b9ffbc3e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d74110e80ba42b896f99b2234bb1ad3, user_id=ac5c9e1e6ce348e39871f20f835028e1, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:24.590204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde5098-906b-48e1-aa5f-f1b9ffbc3e6c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:24.635995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 878/1714] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:24 2022] PUT /identity/v3/projects/9d74110e80ba42b896f99b2234bb1ad3/users/ac5c9e1e6ce348e39871f20f835028e1/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:24.637583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd9c0c-cfda-4d81-a9fe-8b670dd090f1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.637871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd9c0c-cfda-4d81-a9fe-8b670dd090f1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.638161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd9c0c-cfda-4d81-a9fe-8b670dd090f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:24.767976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3bdd9c0c-cfda-4d81-a9fe-8b670dd090f1 None None] MFA Rules not processed for user `ac5c9e1e6ce348e39871f20f835028e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:24.774732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d8cf6-cb6e-4b64-b483-2969f07bf82b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:24.776631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3bdd9c0c-cfda-4d81-a9fe-8b670dd090f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:24.932698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16584cc9-fc7d-4362-8120-64ba1868f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:24.933777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16584cc9-fc7d-4362-8120-64ba1868f1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.934173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16584cc9-fc7d-4362-8120-64ba1868f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.934596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16584cc9-fc7d-4362-8120-64ba1868f1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:24.935439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16584cc9-fc7d-4362-8120-64ba1868f1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:24.939826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16584cc9-fc7d-4362-8120-64ba1868f1d8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:24.971875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 837/1715] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:24.971875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8392cb-aba3-4c23-bf44-98318038543a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:24.971875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8392cb-aba3-4c23-bf44-98318038543a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:24.971875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8392cb-aba3-4c23-bf44-98318038543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:25.006720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 879/1716] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:25.011286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd9c0c-cfda-4d81-a9fe-8b670dd090f1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:25.056729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4c8392cb-aba3-4c23-bf44-98318038543a None None] MFA Rules not processed for user `76b2a2d381a14f5fb04e721905b36ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:25.057998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4c8392cb-aba3-4c23-bf44-98318038543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:25.061207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd9c0c-cfda-4d81-a9fe-8b670dd090f1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:25.079390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:25.080062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:25.080317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:25.080515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] PATH_INFO: `/v3/projects/2aaa2084ed904b08bacb9cc5679a40fb/users/81787c270dce4455831c24d106aaa92f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:25.089024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aaa2084ed904b08bacb9cc5679a40fb, user_id=81787c270dce4455831c24d106aaa92f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:25.101170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:25.143493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 880/1717] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:25 2022] PUT /identity/v3/projects/2aaa2084ed904b08bacb9cc5679a40fb/users/81787c270dce4455831c24d106aaa92f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:25.145317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:25.165866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:25.315218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 838/1718] 10.222.0.25 () {58 vars in 1015 bytes} [Mon May 16 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:25.361235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cc128-f56b-42a3-9619-2fc33e1bccf0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:25.371440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc87a64-6c22-4daa-b9c4-85774701533c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:25.373344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc87a64-6c22-4daa-b9c4-85774701533c service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:25.373648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc87a64-6c22-4daa-b9c4-85774701533c service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:25.373959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc87a64-6c22-4daa-b9c4-85774701533c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:25.444514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8392cb-aba3-4c23-bf44-98318038543a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:25.455416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8392cb-aba3-4c23-bf44-98318038543a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:25.535834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc87a64-6c22-4daa-b9c4-85774701533c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:25.537759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc87a64-6c22-4daa-b9c4-85774701533c service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:25.563235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 881/1719] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:25.606176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:25.607089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:25.607520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:25.608074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:25.629528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:25.631575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:25.653715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 839/1720] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:15:25.737511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:25.738349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:25.738832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:25.796962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] MFA Rules not processed for user `d38cdf40e33b47d8aa79e029cded7503`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:25.824668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:25.964575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:25.976685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52816f-232f-4e43-9723-c072fae6caba service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:25.991115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:25.991935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:25.992670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:25.995049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:26.008567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:26.009129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 882/1721] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:26.024886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:26.037930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:26.250206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35057b6-31ea-4cab-bba5-021449fe5dae None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:26.250206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:26.250206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:26.250206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:26.252064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:26.252350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:26.252350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:26.252520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:26.253097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:26.271405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 840/1722] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:15:26.301436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 883/1723] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:26.770501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:26.772000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:26.886496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:26.888246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:27.012969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e407eb-4d84-411e-922b-fd8107c6d212 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:27.045894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:27.052689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:27.054515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:27.058005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] PATH_INFO: `/v3/projects/ba742161ca6d4c4393f0aa1213c5d4ea` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:27.072216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba742161ca6d4c4393f0aa1213c5d4ea)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:27.080977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:27.125148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30350a3-efce-4cd4-8d11-592728380386 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:27.136688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:27.137593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:27.137878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:27.138157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] PATH_INFO: `/v3/projects/dc83025212a2416ba02124f26c956a04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:27.144623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc83025212a2416ba02124f26c956a04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:27.148676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:27.326633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba742161ca6d4c4393f0aa1213c5d4ea', 'request_id': 'req-e7f5f885-c592-49db-b00b-e87e6d465f06'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.420268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc83025212a2416ba02124f26c956a04', 'request_id': 'req-a07c569e-10fd-4fef-a976-d60c3bfa64a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.432615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba742161ca6d4c4393f0aa1213c5d4ea', 'request_id': 'req-e7f5f885-c592-49db-b00b-e87e6d465f06'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.441631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba742161ca6d4c4393f0aa1213c5d4ea', 'request_id': 'req-e7f5f885-c592-49db-b00b-e87e6d465f06'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.460301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc83025212a2416ba02124f26c956a04', 'request_id': 'req-a07c569e-10fd-4fef-a976-d60c3bfa64a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.470499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] The token cache is being invalidate because project ba742161ca6d4c4393f0aa1213c5d4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:27.471925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.472829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc83025212a2416ba02124f26c956a04', 'request_id': 'req-a07c569e-10fd-4fef-a976-d60c3bfa64a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.478781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f5f885-c592-49db-b00b-e87e6d465f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.480692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 841/1724] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:26 2022] DELETE /identity/v3/projects/ba742161ca6d4c4393f0aa1213c5d4ea => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:27.485330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7caba-9477-46f5-8b43-e0a450179fbd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:27.488915 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7caba-9477-46f5-8b43-e0a450179fbd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:27.489307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7caba-9477-46f5-8b43-e0a450179fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:27.539074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7bf7caba-9477-46f5-8b43-e0a450179fbd None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:27.541543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7bf7caba-9477-46f5-8b43-e0a450179fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:27.556350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] The token cache is being invalidate because project dc83025212a2416ba02124f26c956a04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:27.556684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.560223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a07c569e-10fd-4fef-a976-d60c3bfa64a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:27.570303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 884/1725] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:26 2022] DELETE /identity/v3/projects/dc83025212a2416ba02124f26c956a04 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:27.577569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1447f-7b9d-4b40-8e6c-09cd9bce0621 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:27.578645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1447f-7b9d-4b40-8e6c-09cd9bce0621 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:27.587846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1447f-7b9d-4b40-8e6c-09cd9bce0621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:27.670500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ebf1447f-7b9d-4b40-8e6c-09cd9bce0621 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:27.679164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ebf1447f-7b9d-4b40-8e6c-09cd9bce0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:27.837549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 842/1726] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:27.846662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d84fb-42a2-4315-afd9-c400f89a082a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:27.847763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d84fb-42a2-4315-afd9-c400f89a082a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:27.848247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d84fb-42a2-4315-afd9-c400f89a082a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:27.945935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-870d84fb-42a2-4315-afd9-c400f89a082a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:27.950248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-870d84fb-42a2-4315-afd9-c400f89a082a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:27.959461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 885/1727] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:27.968757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea45e-e015-47f8-aaa8-34f74c5e537f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:27.969062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea45e-e015-47f8-aaa8-34f74c5e537f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:27.969630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea45e-e015-47f8-aaa8-34f74c5e537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:27.992074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 843/1728] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:27.992962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d84fb-42a2-4315-afd9-c400f89a082a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:28.034107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d84fb-42a2-4315-afd9-c400f89a082a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:28.054498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:28.055412 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:28.057973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:28.058248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:28.059377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:28.059852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:28.061719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:28.073932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e10ea45e-e015-47f8-aaa8-34f74c5e537f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:28.075338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e10ea45e-e015-47f8-aaa8-34f74c5e537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:28.101106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 886/1729] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:28.121109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ea45e-e015-47f8-aaa8-34f74c5e537f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:28.124488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 844/1730] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:28.125491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:28.150868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa62d-dad2-4745-9e9d-d4d508d1357d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:28.161998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ea45e-e015-47f8-aaa8-34f74c5e537f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:28.187114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:28.187835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:28.188122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:28.188400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:28.189241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:28.196041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:28.211281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:28.233993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:28.233993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:28.233993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:28.233993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:28.233993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:28.242111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:28.276050 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 887/1731] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:28.280477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:28.329789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292a41-123f-446f-9681-00f173962e38 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:28.348503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:28.350722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:28.350722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:28.350722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:28.350722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:28.368103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:28.388857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 845/1732] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:28 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:28.389795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:28.561830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 888/1733] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:28 2022] POST /identity/v3/projects => generated 371 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:28.564647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:28.780572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361faaf-135d-4045-b755-30c060fc7fd2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:28.859562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b4a73-9972-41a6-ad78-0d453d44ef6e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:28.886933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:28.888461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:28.891986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:28.892279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:28.893025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:28.909311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.004815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:29.005793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.005911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.015218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.017747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:29.020896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.113804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 889/1734] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:28 2022] POST /identity/v3/users => generated 286 bytes in 550 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:29.118349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:29.147258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0eee-0b7d-4a33-9db9-905eeb5747cf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:29.167326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:29.168201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.179051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.179051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.183658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:29.187207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.194498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 846/1735] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:28 2022] POST /identity/v3/users => generated 292 bytes in 805 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:29.195942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:29.231250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 890/1736] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:29.239770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:29.257153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6fc98-f1d2-4970-9011-a677251ed6a8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:29.316548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:29.318926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d1757-20d1-43ec-8d3d-3ee8ed7e5502 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:29.320066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.320695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.321300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.324192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:29.326865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.341465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:29.345458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.349776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.349776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.349776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:29.357130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.378481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 847/1737] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:29.378481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:29.413596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 891/1738] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:29.414575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:29.426043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e28f5-e5c0-499b-99bd-f04348a3a7e5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:29.451282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:29.452020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.452301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.452572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.453836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:29.455061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.457736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad56502-1a0d-4063-a8f0-73519b4a1ca3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:29.476093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05549c-e444-4536-b986-80c9c8a7a003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:29.477007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05549c-e444-4536-b986-80c9c8a7a003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.477285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05549c-e444-4536-b986-80c9c8a7a003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.477554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05549c-e444-4536-b986-80c9c8a7a003 admin admin] PATH_INFO: `/v3/projects/e5344ec4827649e89d6fc6933c739949/users/b5b660247c224c19b1a80411aa14cc9b/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.509262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 848/1739] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:29.513749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:29.541464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05549c-e444-4536-b986-80c9c8a7a003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5344ec4827649e89d6fc6933c739949, user_id=b5b660247c224c19b1a80411aa14cc9b, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:29.545020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05549c-e444-4536-b986-80c9c8a7a003 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.554692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501236d-f723-4b9e-a132-7c37bc751fd1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:29.572500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaac1eb-b460-457c-931a-20ecbbde7d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:29.573411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaac1eb-b460-457c-931a-20ecbbde7d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.573693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaac1eb-b460-457c-931a-20ecbbde7d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.574347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaac1eb-b460-457c-931a-20ecbbde7d79 admin admin] PATH_INFO: `/v3/projects/139c967172e441049771fe04629ca839/users/a7403942423b4de193fd64c56e202659/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.633940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 892/1740] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:29 2022] PUT /identity/v3/projects/e5344ec4827649e89d6fc6933c739949/users/b5b660247c224c19b1a80411aa14cc9b/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:29.646583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa18f36-f136-4bc3-a136-090c091baea6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.652611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa18f36-f136-4bc3-a136-090c091baea6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.652611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa18f36-f136-4bc3-a136-090c091baea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.657245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaac1eb-b460-457c-931a-20ecbbde7d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=139c967172e441049771fe04629ca839, user_id=a7403942423b4de193fd64c56e202659, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:29.660071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaac1eb-b460-457c-931a-20ecbbde7d79 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:29.829720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7aa18f36-f136-4bc3-a136-090c091baea6 None None] MFA Rules not processed for user `b5b660247c224c19b1a80411aa14cc9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:29.834081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7aa18f36-f136-4bc3-a136-090c091baea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:29.857859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 849/1741] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:29 2022] PUT /identity/v3/projects/139c967172e441049771fe04629ca839/users/a7403942423b4de193fd64c56e202659/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:29.859698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:29.859986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:29.860425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:29.968941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] MFA Rules not processed for user `a7403942423b4de193fd64c56e202659`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:29.973828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:30.138073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 893/1742] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:30.139297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa18f36-f136-4bc3-a136-090c091baea6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:30.236268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 850/1743] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:30.247538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:30.249171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:30.358320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa18f36-f136-4bc3-a136-090c091baea6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:30.460351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:30.461380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:30.461855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:30.462284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:30.463130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:30.465554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:30.470103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbe241-5822-47c5-a1e9-643fb4dbe8d8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:30.488442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:30.491681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:30.491974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:30.492252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:30.534909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:30.536507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:30.569152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 851/1744] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:15:30.632365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 894/1745] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:30 2022] POST /identity/v3/projects => generated 371 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:30.633619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:30.655988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a212ed-c712-4a0f-9a2e-cd0f15b5c0f1 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:30.835254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:30.836005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:30.836297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:30.836574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:30.837346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:30.851068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:30.989910 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 852/1746] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:30 2022] POST /identity/v3/users => generated 292 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:30.994198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:31.025940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a9466-b370-442b-bb80-15059e644bd7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:31.059485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:31.060235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:31.063558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:31.063995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:31.065325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:31.071955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:31.087356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 895/1747] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:31.091500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:31.108588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaf742-9c06-4f07-9dd8-d59a98b2e340 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:31.120420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:31.121406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:31.121595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:31.121881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] PATH_INFO: `/v3/projects/ddac248ba7f94ab9982e6d932561b12e/users/02427714a78741c39b1e96884a5b4619/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:31.176677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddac248ba7f94ab9982e6d932561b12e, user_id=02427714a78741c39b1e96884a5b4619, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:31.186332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:31.295226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 853/1748] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:31 2022] PUT /identity/v3/projects/ddac248ba7f94ab9982e6d932561b12e/users/02427714a78741c39b1e96884a5b4619/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:31.298955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:31.333374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965734f-5f35-46ea-8f53-9678850f8a70 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:31.476889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcda46-b67f-4565-8f82-4ded1dcab1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:31.478448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcda46-b67f-4565-8f82-4ded1dcab1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:31.478952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcda46-b67f-4565-8f82-4ded1dcab1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:31.479439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcda46-b67f-4565-8f82-4ded1dcab1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:31.480746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcda46-b67f-4565-8f82-4ded1dcab1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:31.485699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcda46-b67f-4565-8f82-4ded1dcab1a6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:31.511344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 896/1749] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:31.515925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:31.528902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb75c05-b250-457a-bd57-b0c429552c21 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:31.535893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec6a1d-a82a-4387-a855-3c8ebaad6ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:31.537319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6a1d-a82a-4387-a855-3c8ebaad6ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:31.537528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6a1d-a82a-4387-a855-3c8ebaad6ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:31.538031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6a1d-a82a-4387-a855-3c8ebaad6ad6 admin admin] PATH_INFO: `/v3/projects/ddac248ba7f94ab9982e6d932561b12e/users/02427714a78741c39b1e96884a5b4619/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:31.541063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec6a1d-a82a-4387-a855-3c8ebaad6ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddac248ba7f94ab9982e6d932561b12e, user_id=02427714a78741c39b1e96884a5b4619, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:31.545193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec6a1d-a82a-4387-a855-3c8ebaad6ad6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:31.569892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 854/1750] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:31 2022] PUT /identity/v3/projects/ddac248ba7f94ab9982e6d932561b12e/users/02427714a78741c39b1e96884a5b4619/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:31.580434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:31.581190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:31.582550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:31.655705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] MFA Rules not processed for user `02427714a78741c39b1e96884a5b4619`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:31.658868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:31.717983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7355-6168-4039-bf13-c332dd68930f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:31.718281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7355-6168-4039-bf13-c332dd68930f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:31.718554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7355-6168-4039-bf13-c332dd68930f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:31.806753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c0cf7355-6168-4039-bf13-c332dd68930f None None] MFA Rules not processed for user `7b559cc982a846bea7f7f2077b7c455b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:31.806753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c0cf7355-6168-4039-bf13-c332dd68930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:32.071905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 855/1751] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:32.257590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 897/1752] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 678 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:32.262250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd1a5-5a01-4e6f-ab26-e0ac96ce0f6d None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:32.262888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd1a5-5a01-4e6f-ab26-e0ac96ce0f6d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:32.263411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd1a5-5a01-4e6f-ab26-e0ac96ce0f6d None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:32.265150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 856/1753] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:32.273321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:32.278026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:32.380992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd1a5-5a01-4e6f-ab26-e0ac96ce0f6d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:32.382847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-100bd1a5-5a01-4e6f-ab26-e0ac96ce0f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:32.486327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a3f74-64a8-4ed4-865e-a8065658c43c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:32.499722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529694a-5ce5-4224-a846-d5bc2cd91255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:32.505130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529694a-5ce5-4224-a846-d5bc2cd91255 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:32.505507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529694a-5ce5-4224-a846-d5bc2cd91255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:32.505900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529694a-5ce5-4224-a846-d5bc2cd91255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:32.534712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529694a-5ce5-4224-a846-d5bc2cd91255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:32.537017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529694a-5ce5-4224-a846-d5bc2cd91255 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:32.588775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 898/1754] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:32.590912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:32.591753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:32.592715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:32.628376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd1a5-5a01-4e6f-ab26-e0ac96ce0f6d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:32.636946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:32.638979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:32.644865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9800f7-c04f-417f-beaf-94888d9b6fe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:32.645882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9800f7-c04f-417f-beaf-94888d9b6fe4 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:32.646242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9800f7-c04f-417f-beaf-94888d9b6fe4 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:32.646577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9800f7-c04f-417f-beaf-94888d9b6fe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:32.678584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9800f7-c04f-417f-beaf-94888d9b6fe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:32.682055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9800f7-c04f-417f-beaf-94888d9b6fe4 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:32.682302 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 899/1755] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:15:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:32.697444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 857/1756] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:32.726260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:32.729445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:32.794629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff6f3a-1d82-4058-984c-5ce6cad94aed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:32.812891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:32.813927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:32.814402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:32.814907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:32.839812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:32.859414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:32.885483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 900/1757] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:33.212385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9c59-aa04-4052-9e26-3915b8c392ed None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:33.212640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9c59-aa04-4052-9e26-3915b8c392ed None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:33.212921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9c59-aa04-4052-9e26-3915b8c392ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:33.252522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-258c9c59-aa04-4052-9e26-3915b8c392ed None None] MFA Rules not processed for user `81787c270dce4455831c24d106aaa92f`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:33.259805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-258c9c59-aa04-4052-9e26-3915b8c392ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:33.334742 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 858/1758] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:33.832324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:33.942008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a2e55-c86a-4a92-af4d-d06c10521df8 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:33.948059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:33.949638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:33.950003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:33.950678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] PATH_INFO: `/v3/users/b5b660247c224c19b1a80411aa14cc9b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:33.952426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b660247c224c19b1a80411aa14cc9b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:33.954338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:34.275508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b660247c224c19b1a80411aa14cc9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:34.353440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b660247c224c19b1a80411aa14cc9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:34.363061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b660247c224c19b1a80411aa14cc9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:34.364450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b660247c224c19b1a80411aa14cc9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:34.401232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b660247c224c19b1a80411aa14cc9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:34.410615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 901/1759] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:33 2022] DELETE /identity/v3/users/b5b660247c224c19b1a80411aa14cc9b => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:34.417227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43aa66-d45f-4051-a234-d7c01797ab78 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:34.417813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43aa66-d45f-4051-a234-d7c01797ab78 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:34.417813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43aa66-d45f-4051-a234-d7c01797ab78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:34.446899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4f43aa66-d45f-4051-a234-d7c01797ab78 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:34.448422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4f43aa66-d45f-4051-a234-d7c01797ab78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:34.538054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 859/1760] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:34.582509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:34.585521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:34.708976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdfdce-4608-43d3-8927-1c89c68166e9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:34.715465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:34.718694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:34.718694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:34.718694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:34.733840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:34.737527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:34.755450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 902/1761] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:36.246029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d99c9-fdb3-484a-9b8f-96c9fbfde4eb None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:36.246581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d99c9-fdb3-484a-9b8f-96c9fbfde4eb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:36.246581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d99c9-fdb3-484a-9b8f-96c9fbfde4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:36.336285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ee3d99c9-fdb3-484a-9b8f-96c9fbfde4eb None None] MFA Rules not processed for user `d38cdf40e33b47d8aa79e029cded7503`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:36.337476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ee3d99c9-fdb3-484a-9b8f-96c9fbfde4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:36.530188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 860/1762] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:36.976479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:36.979782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:37.015101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa79b4-005b-43c4-a160-43944b3986dd service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:37.029920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:37.030862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:37.031148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:37.031429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] PATH_INFO: `/v3/projects/e5344ec4827649e89d6fc6933c739949` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:37.039338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5344ec4827649e89d6fc6933c739949)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:37.044737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:37.256548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5344ec4827649e89d6fc6933c739949', 'request_id': 'req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:37.266414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5344ec4827649e89d6fc6933c739949', 'request_id': 'req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:37.297293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5344ec4827649e89d6fc6933c739949', 'request_id': 'req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:37.400123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] The token cache is being invalidate because project e5344ec4827649e89d6fc6933c739949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:37.401460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:37.403944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:37.411753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 903/1763] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:36 2022] DELETE /identity/v3/projects/e5344ec4827649e89d6fc6933c739949 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:37.412499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d99c9-fdb3-484a-9b8f-96c9fbfde4eb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:37.453447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d99c9-fdb3-484a-9b8f-96c9fbfde4eb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:37.617759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:37.619922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:37.620339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:37.620743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] PATH_INFO: `/v3/users/02427714a78741c39b1e96884a5b4619` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:37.622195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=02427714a78741c39b1e96884a5b4619)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:37.624445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:37.893126 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02427714a78741c39b1e96884a5b4619'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:38.017687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02427714a78741c39b1e96884a5b4619'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:38.017687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02427714a78741c39b1e96884a5b4619'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:38.034310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02427714a78741c39b1e96884a5b4619'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:38.048629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a54de6ee-aa33-4d25-bcf9-c981e45345ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02427714a78741c39b1e96884a5b4619'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:38.065215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 861/1764] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:37 2022] DELETE /identity/v3/users/02427714a78741c39b1e96884a5b4619 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:38.464092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:38.465756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:38.496182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b3a2-ee9d-4779-bb00-1e0ba78b01ad admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:38.592884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:38.598475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:38.598475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:38.598475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] PATH_INFO: `/v3/users/2a614c90163d425a9b04ed8574bb2a91` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:38.598475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a614c90163d425a9b04ed8574bb2a91)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:38.599306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:38.725340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892ee73-38e9-4d2d-8264-16b5cb06e895 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:38.726502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892ee73-38e9-4d2d-8264-16b5cb06e895 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:38.726502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892ee73-38e9-4d2d-8264-16b5cb06e895 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:38.727315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 862/1765] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:38.748539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0bb3-6ca7-4248-ab1c-fcdb2d632b3d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:38.749081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0bb3-6ca7-4248-ab1c-fcdb2d632b3d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:38.749538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0bb3-6ca7-4248-ab1c-fcdb2d632b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:38.784346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e8aa0bb3-6ca7-4248-ab1c-fcdb2d632b3d None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:38.789591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e8aa0bb3-6ca7-4248-ab1c-fcdb2d632b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:38.873594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a614c90163d425a9b04ed8574bb2a91'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:38.922460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 863/1766] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:15:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:38.922460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:38.922460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:38.922460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:38.944191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] MFA Rules not processed for user `7b559cc982a846bea7f7f2077b7c455b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:38.945413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:38.991792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a614c90163d425a9b04ed8574bb2a91'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.009207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a614c90163d425a9b04ed8574bb2a91'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.010061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a614c90163d425a9b04ed8574bb2a91'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.028920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a614c90163d425a9b04ed8574bb2a91'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.047526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 904/1767] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:38 2022] DELETE /identity/v3/users/2a614c90163d425a9b04ed8574bb2a91 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:39.048543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:39.059315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:39.111860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 864/1768] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:39.113771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:39.119396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:39.167031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9040b-9dfe-4e2e-b6de-0d8ad03f9eab None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:39.211713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70877dc2-4240-4c21-be62-d0a3f751fe0b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:39.217610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7728-de6a-4987-8357-456246f445ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:39.218661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7728-de6a-4987-8357-456246f445ba service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.218661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7728-de6a-4987-8357-456246f445ba service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.218851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7728-de6a-4987-8357-456246f445ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.220827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-deab7728-de6a-4987-8357-456246f445ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:39.234518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab7728-de6a-4987-8357-456246f445ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:39.236978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab7728-de6a-4987-8357-456246f445ba service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:39.262233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 905/1769] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:39.263326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:39.264543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.264817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.265133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] PATH_INFO: `/v3/projects/ddac248ba7f94ab9982e6d932561b12e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.266620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddac248ba7f94ab9982e6d932561b12e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:39.273278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:39.409271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddac248ba7f94ab9982e6d932561b12e', 'request_id': 'req-77ccb44e-eefa-466f-9037-d94000ecb64a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.494424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7728-de6a-4987-8357-456246f445ba service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:39.496026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-deab7728-de6a-4987-8357-456246f445ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:39.511788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7728-de6a-4987-8357-456246f445ba service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:39.512660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddac248ba7f94ab9982e6d932561b12e', 'request_id': 'req-77ccb44e-eefa-466f-9037-d94000ecb64a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.528841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddac248ba7f94ab9982e6d932561b12e', 'request_id': 'req-77ccb44e-eefa-466f-9037-d94000ecb64a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.546250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] The token cache is being invalidate because project ddac248ba7f94ab9982e6d932561b12e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:39.552161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.552161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77ccb44e-eefa-466f-9037-d94000ecb64a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.553963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 865/1770] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:39 2022] DELETE /identity/v3/projects/ddac248ba7f94ab9982e6d932561b12e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:39.555514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b2caa-c13f-4354-a611-6dc118a4b824 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.555933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b2caa-c13f-4354-a611-6dc118a4b824 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.556998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b2caa-c13f-4354-a611-6dc118a4b824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.580873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3e1b2caa-c13f-4354-a611-6dc118a4b824 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:39.580873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3e1b2caa-c13f-4354-a611-6dc118a4b824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:39.594113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:39.595336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.595845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.596113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] PATH_INFO: `/v3/projects/3f804032882d4ed7ba65b5969dd00ea8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.598083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f804032882d4ed7ba65b5969dd00ea8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:39.600513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:39.675971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 866/1771] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:39.677784 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fbbc8-ca59-4547-9525-01c39b8100e1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.678150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fbbc8-ca59-4547-9525-01c39b8100e1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.679671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fbbc8-ca59-4547-9525-01c39b8100e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.721227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7f9fbbc8-ca59-4547-9525-01c39b8100e1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:39.723265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7f9fbbc8-ca59-4547-9525-01c39b8100e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:39.731320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f804032882d4ed7ba65b5969dd00ea8', 'request_id': 'req-6fd7e47f-8104-4a20-bd00-28908524e782'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.736854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f804032882d4ed7ba65b5969dd00ea8', 'request_id': 'req-6fd7e47f-8104-4a20-bd00-28908524e782'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.745101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f804032882d4ed7ba65b5969dd00ea8', 'request_id': 'req-6fd7e47f-8104-4a20-bd00-28908524e782'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.786868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 867/1772] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:39.791466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fbbc8-ca59-4547-9525-01c39b8100e1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:39.801387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fbbc8-ca59-4547-9525-01c39b8100e1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:39.806388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:39.806989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.807310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.807593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.808389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:39.808695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:39.810328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:39.825302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 868/1773] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:39.826434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:39.827584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] The token cache is being invalidate because project 3f804032882d4ed7ba65b5969dd00ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:39.827975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.828773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:39.834061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 906/1774] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:39 2022] DELETE /identity/v3/projects/3f804032882d4ed7ba65b5969dd00ea8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:15:39.834948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:39.836386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:39.837976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aeaa9-dd07-4983-b216-0053a2924284 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:39.846584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:39.848138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.849225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.849225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.849833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:39.850538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7e47f-8104-4a20-bd00-28908524e782 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:39.852688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:39.855833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:39.856861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:39.857293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:39.857713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] PATH_INFO: `/v3/users/81787c270dce4455831c24d106aaa92f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:39.859410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81787c270dce4455831c24d106aaa92f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:39.861606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:39.933982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 869/1775] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:39 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:39.937909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:39.939618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:40.002628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ce60d-5dec-4ac7-8993-dc44fdbf1277 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:40.008632 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:40.010288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:40.017281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:40.017281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:40.017281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:40.017281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:40.101873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81787c270dce4455831c24d106aaa92f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.145725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 870/1776] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:39 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:40.149068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:40.160754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5c79-ea20-4e4e-aec3-f1aaf40ec251 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:40.165821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:40.166612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:40.166965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:40.167240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:40.168019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:40.169861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:40.181676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 871/1777] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:40.185349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:40.198147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a68ea4-bd97-4f86-982b-fc89187ab130 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:40.203506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:40.205578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:40.205578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:40.205578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:40.205807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:40.208164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:40.218853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 872/1778] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:40.224581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:40.229400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81787c270dce4455831c24d106aaa92f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.240435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec997fc4-539c-4ba1-b3cd-26aeee88bbd7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:40.242667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81787c270dce4455831c24d106aaa92f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.242667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81787c270dce4455831c24d106aaa92f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.246453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:40.247596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:40.247910 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:40.248213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] PATH_INFO: `/v3/projects/b2960247ce954c1bbc611ff54b9f461c/users/ee59d444570e45de81e5cabe308311d0/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:40.257448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-19925efb-6c08-4031-819e-e4b8526b7eb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81787c270dce4455831c24d106aaa92f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.264680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 907/1779] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:39 2022] DELETE /identity/v3/users/81787c270dce4455831c24d106aaa92f => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:40.291829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2960247ce954c1bbc611ff54b9f461c, user_id=ee59d444570e45de81e5cabe308311d0, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:40.297243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:40.443037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 873/1780] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:40 2022] PUT /identity/v3/projects/b2960247ce954c1bbc611ff54b9f461c/users/ee59d444570e45de81e5cabe308311d0/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:40.453404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:40.454485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:40.454916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:40.517359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] MFA Rules not processed for user `ee59d444570e45de81e5cabe308311d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:40.520412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:40.641792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 908/1781] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:40.646419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:40.648677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:40.673648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:40.680985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:40.771243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860bbfb-b5e3-41be-ad70-1178a1e5608f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:40.778540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:40.779578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:40.779988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:40.780259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:40.780963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:40.782877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:40.793334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0ed28-6ae3-46ba-8d60-5479f4b591cd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:40.799008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:40.800651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:40.800993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:40.801888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] PATH_INFO: `/v3/projects/2aaa2084ed904b08bacb9cc5679a40fb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:40.803375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aaa2084ed904b08bacb9cc5679a40fb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:40.805491 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:40.891680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 874/1782] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:40 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:40.894581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:40.954619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aaa2084ed904b08bacb9cc5679a40fb', 'request_id': 'req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.962290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aaa2084ed904b08bacb9cc5679a40fb', 'request_id': 'req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.969716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aaa2084ed904b08bacb9cc5679a40fb', 'request_id': 'req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:40.984079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df202a-2816-4505-a756-1a3cd56f0003 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.047285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.048089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.048452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.048787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.049141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] The token cache is being invalidate because project 2aaa2084ed904b08bacb9cc5679a40fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:41.049444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:41.049672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.050739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-875a0dbc-e3a1-4cae-a9ed-70ca660ea000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:41.052975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.057239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 909/1783] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:40 2022] DELETE /identity/v3/projects/2aaa2084ed904b08bacb9cc5679a40fb => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:41.058706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d9a22-0bae-44b0-b1d2-cb7b895f9087 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.058985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d9a22-0bae-44b0-b1d2-cb7b895f9087 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.059267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d9a22-0bae-44b0-b1d2-cb7b895f9087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.089516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7d4d9a22-0bae-44b0-b1d2-cb7b895f9087 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:41.091097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d4d9a22-0bae-44b0-b1d2-cb7b895f9087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:41.108608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 910/1784] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:41.110293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bc29a-7d5d-4479-9514-ae1384b14f89 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.110585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bc29a-7d5d-4479-9514-ae1384b14f89 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.110863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bc29a-7d5d-4479-9514-ae1384b14f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.138113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 875/1785] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:40 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:41.142465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.144435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:41.159315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f26ed0-97db-48fb-ad0d-d934c6dbe21f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.163383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c21bc29a-7d5d-4479-9514-ae1384b14f89 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:41.165873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c21bc29a-7d5d-4479-9514-ae1384b14f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:41.165967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.166650 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.166937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.167207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.168099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.169772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.183186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 876/1786] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:41.187419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.197599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 911/1787] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:41.204198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bc29a-7d5d-4479-9514-ae1384b14f89 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.205161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f22cc-606b-4573-99e2-dcdd014e46d1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.211315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.212323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.217544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.217544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] PATH_INFO: `/v3/projects/4ba02b377c734d36bc7f1fc8cd670ea8/users/5f941f846c8448d296923c2a8d786ea7/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.256677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba02b377c734d36bc7f1fc8cd670ea8, user_id=5f941f846c8448d296923c2a8d786ea7, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.260260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bc29a-7d5d-4479-9514-ae1384b14f89 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.263472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.275140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.276216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.276702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.277210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.278505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:41.278939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.280736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.301144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 912/1788] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:41.302842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.326192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84b7c6-1c1f-415f-a33b-61b6e999b3cb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.331748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.332651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.333468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.336109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.341680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.341924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 877/1789] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:41 2022] PUT /identity/v3/projects/4ba02b377c734d36bc7f1fc8cd670ea8/users/5f941f846c8448d296923c2a8d786ea7/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:41.344815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.348744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.501503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 913/1790] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:41 2022] POST /identity/v3/projects => generated 401 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:41.502978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fa5f-22bc-4391-89c7-91b16b308067 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.509164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.606800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2982d-a3a9-4773-a7e0-9c67635f1269 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.613800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.614908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.615252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.615543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.616563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.619017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.683852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.685157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.686769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.688044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.690390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.696499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.729654 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 878/1791] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:41.731240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.754626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 914/1792] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:41 2022] POST /identity/v3/users => generated 301 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:41.756293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fcadb-087c-4bfd-98ec-58664ef14fe4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.758710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.776326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14d79e-352d-47cb-8c0b-505190e2850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.786039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14d79e-352d-47cb-8c0b-505190e2850c admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.789521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14d79e-352d-47cb-8c0b-505190e2850c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.794264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14d79e-352d-47cb-8c0b-505190e2850c admin admin] PATH_INFO: `/v3/projects/4ba02b377c734d36bc7f1fc8cd670ea8/users/5f941f846c8448d296923c2a8d786ea7/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.797896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14d79e-352d-47cb-8c0b-505190e2850c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba02b377c734d36bc7f1fc8cd670ea8, user_id=5f941f846c8448d296923c2a8d786ea7, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.802625 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14d79e-352d-47cb-8c0b-505190e2850c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.803900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5289a-f4c7-4064-8b15-d9914faee86c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:41.815151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:41.816901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.817377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.817988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.820448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:41.822280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 879/1793] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:41 2022] PUT /identity/v3/projects/4ba02b377c734d36bc7f1fc8cd670ea8/users/5f941f846c8448d296923c2a8d786ea7/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:41.823757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:41.833797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:41.835474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:41.836073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:41.881943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] MFA Rules not processed for user `5f941f846c8448d296923c2a8d786ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:41.881943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:41.889297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 915/1794] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:41.898230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:41.929139 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576c2b-6f18-423f-a305-bbfb6038666e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:42.110410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 880/1795] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:42.118043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:42.120315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:42.120621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:42.120968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:42.121772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:42.128138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:42.129510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:42.131927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:42.150637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 916/1796] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:42.156386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:42.174378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9eca1-5a8e-4bb4-a90c-8d968a04d669 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:42.198732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda2018-58d8-4850-aa7b-0285b624c237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:42.207732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda2018-58d8-4850-aa7b-0285b624c237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:42.207732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda2018-58d8-4850-aa7b-0285b624c237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:42.207732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda2018-58d8-4850-aa7b-0285b624c237 admin admin] PATH_INFO: `/v3/projects/3af9e1872f144599807c605bff21f770/users/02a5d450f56d41ab95da3024a68d172a/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:42.264675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda2018-58d8-4850-aa7b-0285b624c237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af9e1872f144599807c605bff21f770, user_id=02a5d450f56d41ab95da3024a68d172a, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:42.275700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda2018-58d8-4850-aa7b-0285b624c237 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:42.320028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a0c1b-8673-4301-909b-abe043b1bf30 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:42.325440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:42.326202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:42.326611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:42.327222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:42.357963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:42.360513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:42.392057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 917/1797] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:42 2022] PUT /identity/v3/projects/3af9e1872f144599807c605bff21f770/users/02a5d450f56d41ab95da3024a68d172a/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:42.392057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033b62-8df6-4c91-b2e4-0465f65ddbac None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:42.392057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033b62-8df6-4c91-b2e4-0465f65ddbac None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:42.392057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033b62-8df6-4c91-b2e4-0465f65ddbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:42.393510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 881/1798] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:42 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:15:42.444215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-77033b62-8df6-4c91-b2e4-0465f65ddbac None None] MFA Rules not processed for user `02a5d450f56d41ab95da3024a68d172a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:42.445403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-77033b62-8df6-4c91-b2e4-0465f65ddbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:42.559939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 918/1799] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:42.560546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:42.577375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97420da7-d476-4da6-a0ee-5c5dcc081670 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:42.757746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033b62-8df6-4c91-b2e4-0465f65ddbac None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:42.758165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:42.758165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:42.758165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:42.758165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:42.758509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:42.758509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:42.787566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033b62-8df6-4c91-b2e4-0465f65ddbac None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:42.792828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:42.793669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:42.794272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:42.794660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:42.797177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:42.891790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 882/1800] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:42 2022] POST /identity/v3/projects => generated 401 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:42.927794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:42.927794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:42.928494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 919/1801] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:42.931469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:43.079739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44e63c-111f-45db-a033-1a2580c65877 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:43.087945 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:43.091030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:43.091691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:43.092554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:43.093656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:43.100671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:43.256779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 883/1802] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:42 2022] POST /identity/v3/users => generated 307 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:43.263523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:43.295144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7e383-d2f5-4c28-8659-bf6b345702a5 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:43.305816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:43.306670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:43.307266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:43.307992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:43.309192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:43.317957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:43.327246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 920/1803] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:43.333150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:43.347518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5aa46-ffa7-47c0-b78c-3a129d520e50 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:43.354912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:43.363439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:43.364065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:43.364363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] PATH_INFO: `/v3/projects/0560e49f7b694d7998fe6df0d7ecc358/users/3fba0a4a385940e9a58777acfd73cbb2/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:43.390158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0560e49f7b694d7998fe6df0d7ecc358, user_id=3fba0a4a385940e9a58777acfd73cbb2, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:43.392996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:43.497944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 884/1804] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:43 2022] PUT /identity/v3/projects/0560e49f7b694d7998fe6df0d7ecc358/users/3fba0a4a385940e9a58777acfd73cbb2/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:43.501965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:43.513024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124cb33-0b33-4148-a036-1b9f192dacd4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:43.569306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef1931-27e0-458e-babe-a3b155612b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:43.570086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef1931-27e0-458e-babe-a3b155612b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:43.570371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef1931-27e0-458e-babe-a3b155612b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:43.570647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef1931-27e0-458e-babe-a3b155612b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:43.571377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef1931-27e0-458e-babe-a3b155612b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:43.574350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef1931-27e0-458e-babe-a3b155612b47 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:43.584719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 921/1805] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:43.588803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:43.600719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa85c7a-505a-43a8-bec8-008c2ab8c64d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:43.637867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:43.641219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:43.658305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:43.658599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] PATH_INFO: `/v3/projects/0560e49f7b694d7998fe6df0d7ecc358/users/3fba0a4a385940e9a58777acfd73cbb2/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:43.664082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0560e49f7b694d7998fe6df0d7ecc358, user_id=3fba0a4a385940e9a58777acfd73cbb2, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:43.666980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:43.692955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 885/1806] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:43 2022] PUT /identity/v3/projects/0560e49f7b694d7998fe6df0d7ecc358/users/3fba0a4a385940e9a58777acfd73cbb2/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:43.698745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:43.699052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:43.699273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:43.741830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] MFA Rules not processed for user `3fba0a4a385940e9a58777acfd73cbb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:43.742527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:43.823454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 922/1807] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:43.898021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:43.955909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eac94c-f0d4-4f04-9a24-1d04b32f2179 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:43.959176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:43.960647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:43.960823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:43.961456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:43.961782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:43.962010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:43.973651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:43.977808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:43.989470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 886/1808] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:43 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:44.044354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eeb70a-91e3-4096-99e6-470b8128cd54 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:44.048921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:44.049793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:44.050250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:44.050699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] PATH_INFO: `/v3/users/cf50192ebb4f4de29739b6485674129c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:44.051857 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf50192ebb4f4de29739b6485674129c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:44.053782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:44.289303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:44.296602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf50192ebb4f4de29739b6485674129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:44.298536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3a91b-2d06-451d-9f67-1f296d3ae8cb service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:44.303767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:44.304544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:44.304837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:44.305117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:44.373742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf50192ebb4f4de29739b6485674129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:44.383800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf50192ebb4f4de29739b6485674129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:44.384508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf50192ebb4f4de29739b6485674129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:44.398360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:44.400071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:44.400558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70734994-a049-4538-82c8-eaf6c640b4a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf50192ebb4f4de29739b6485674129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:44.408410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 923/1809] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:43 2022] DELETE /identity/v3/users/cf50192ebb4f4de29739b6485674129c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:44.413781 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:44.414151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:44.414576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:44.429269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 887/1810] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:44.455526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:44.457833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:44.546405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 924/1811] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:44.556926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:44.558678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:44.626250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87270e4e-7833-4e8b-bff1-7bdb93d7f6d8 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:44.678611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:44.679331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:44.679601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:44.679943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:44.698584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:44.700315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:44.712306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 888/1812] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:45.220381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:45.224779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:45.245521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbe648-83aa-4d35-b04f-7212938b59b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:45.250948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:45.256167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:45.256167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:45.256167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] PATH_INFO: `/v3/projects/57148ac591604d0f8e2bf158675f5f11` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:45.256167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57148ac591604d0f8e2bf158675f5f11)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:45.265362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:45.400766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57148ac591604d0f8e2bf158675f5f11', 'request_id': 'req-2e90e1cd-93a7-483e-8628-674c6ef96877'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.425020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57148ac591604d0f8e2bf158675f5f11', 'request_id': 'req-2e90e1cd-93a7-483e-8628-674c6ef96877'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.433213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57148ac591604d0f8e2bf158675f5f11', 'request_id': 'req-2e90e1cd-93a7-483e-8628-674c6ef96877'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.527673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] The token cache is being invalidate because project 57148ac591604d0f8e2bf158675f5f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:45.527673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.527673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.527673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 925/1813] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:45 2022] DELETE /identity/v3/projects/57148ac591604d0f8e2bf158675f5f11 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:45.530148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:45.545049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2c55a-f697-406d-9bee-34d109fa6162 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:45.669866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:45.670741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:45.671072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:45.671289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] PATH_INFO: `/v3/users/ac5c9e1e6ce348e39871f20f835028e1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:45.672491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5c9e1e6ce348e39871f20f835028e1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:45.674515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:45.874849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5c9e1e6ce348e39871f20f835028e1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.973276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5c9e1e6ce348e39871f20f835028e1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.974263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5c9e1e6ce348e39871f20f835028e1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:45.990386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5c9e1e6ce348e39871f20f835028e1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:46.000500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5c9e1e6ce348e39871f20f835028e1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:46.008051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 889/1814] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:45 2022] DELETE /identity/v3/users/ac5c9e1e6ce348e39871f20f835028e1 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:46.513639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:46.515058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:46.529740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90e1cd-93a7-483e-8628-674c6ef96877 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:46.595574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:46.596712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:46.597180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:46.597658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] PATH_INFO: `/v3/users/a7403942423b4de193fd64c56e202659` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:46.599701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7403942423b4de193fd64c56e202659)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:46.601660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:46.885863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7403942423b4de193fd64c56e202659'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:46.887226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:46.889256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:46.907872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411fe83-874d-4246-87a8-d5b659020afd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:46.921402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:46.922707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:46.922870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:46.923153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] PATH_INFO: `/v3/projects/9d74110e80ba42b896f99b2234bb1ad3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:46.924599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d74110e80ba42b896f99b2234bb1ad3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:46.928300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:46.954894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7403942423b4de193fd64c56e202659'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:46.965564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7403942423b4de193fd64c56e202659'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:46.968083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7403942423b4de193fd64c56e202659'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:46.985140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-08fa72c8-3b5f-4020-9a32-d29c94532a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7403942423b4de193fd64c56e202659'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:46.994453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 926/1815] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:46 2022] DELETE /identity/v3/users/a7403942423b4de193fd64c56e202659 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:47.001710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.002296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.002726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.032509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:47.035090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.110162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d74110e80ba42b896f99b2234bb1ad3', 'request_id': 'req-37fb1477-94d4-4505-9474-bbae72b623b9'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.118822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 927/1816] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:47.120489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.121852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.135438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16c34b-d0a3-4ac2-b4c5-629a9f4d4dc3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.200264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.202704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.202704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.202704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] PATH_INFO: `/v3/users/ee59d444570e45de81e5cabe308311d0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.203742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee59d444570e45de81e5cabe308311d0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.207061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.221653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d74110e80ba42b896f99b2234bb1ad3', 'request_id': 'req-37fb1477-94d4-4505-9474-bbae72b623b9'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.234029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d74110e80ba42b896f99b2234bb1ad3', 'request_id': 'req-37fb1477-94d4-4505-9474-bbae72b623b9'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.257974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] The token cache is being invalidate because project 9d74110e80ba42b896f99b2234bb1ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:47.257974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.257974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.257974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 890/1817] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:46 2022] DELETE /identity/v3/projects/9d74110e80ba42b896f99b2234bb1ad3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:47.257974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.259213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.323382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb1477-94d4-4505-9474-bbae72b623b9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.327188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0031510-7194-4da8-9a1e-bf0be5f62557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.327928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0031510-7194-4da8-9a1e-bf0be5f62557 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.328206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0031510-7194-4da8-9a1e-bf0be5f62557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.328473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0031510-7194-4da8-9a1e-bf0be5f62557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.330234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d0031510-7194-4da8-9a1e-bf0be5f62557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.354396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0031510-7194-4da8-9a1e-bf0be5f62557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.356066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0031510-7194-4da8-9a1e-bf0be5f62557 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.370708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 891/1818] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:47.372038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265fb50-f75b-46f4-898a-3601c46730f1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.372537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265fb50-f75b-46f4-898a-3601c46730f1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.373102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265fb50-f75b-46f4-898a-3601c46730f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.399733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1265fb50-f75b-46f4-898a-3601c46730f1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:47.400927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1265fb50-f75b-46f4-898a-3601c46730f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.412349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 892/1819] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:47.420844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6529f-241c-44de-bb9c-d9b34689bac6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.421147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6529f-241c-44de-bb9c-d9b34689bac6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.421395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6529f-241c-44de-bb9c-d9b34689bac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.442922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5eb6529f-241c-44de-bb9c-d9b34689bac6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:47.444751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee59d444570e45de81e5cabe308311d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.444873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5eb6529f-241c-44de-bb9c-d9b34689bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.454560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 893/1820] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:47.459476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6529f-241c-44de-bb9c-d9b34689bac6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.469302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6529f-241c-44de-bb9c-d9b34689bac6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.475114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.475826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.476148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.477110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.478419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:47.478419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.480119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.494562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 894/1821] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:47.498979 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.510874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a5629-f29f-468b-8e05-bd327ef988ff admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.515551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.518389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.518389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.518389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.518389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.519963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.537384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee59d444570e45de81e5cabe308311d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.544794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee59d444570e45de81e5cabe308311d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.545579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee59d444570e45de81e5cabe308311d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.559775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c93f3cc2-d5a4-4b21-a6a2-3ebfd028b0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee59d444570e45de81e5cabe308311d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:47.570287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 928/1822] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:47 2022] DELETE /identity/v3/users/ee59d444570e45de81e5cabe308311d0 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:47.573951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23694041-a065-47df-bf42-09a5c156f84c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.574213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23694041-a065-47df-bf42-09a5c156f84c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.574827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23694041-a065-47df-bf42-09a5c156f84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.601632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-23694041-a065-47df-bf42-09a5c156f84c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:47.602947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23694041-a065-47df-bf42-09a5c156f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.610292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 895/1823] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:47 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:47.615170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.617912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.699731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 929/1824] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:47.739314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23694041-a065-47df-bf42-09a5c156f84c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.739498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691da797-9c56-44cf-acb6-4c458556ffe8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.742773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23694041-a065-47df-bf42-09a5c156f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.745012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.746258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.746258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.746446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.747231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.749359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.829379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23694041-a065-47df-bf42-09a5c156f84c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.857430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 896/1825] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:47 2022] POST /identity/v3/users => generated 291 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:47.861344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.876671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440ef55-d368-41f2-b26a-5f29228be6d5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.883247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.884329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.884478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.885030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.885874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.888251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.900288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 897/1826] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:47.904053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.904330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.905035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.905406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.905752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.916725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cc1c-84f4-4d6f-9625-59b7280b0243 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.921308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.921308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.922271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.923274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.923798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.924173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.925037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.926738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.930164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 930/1827] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:47.931254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.932859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:47.938877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 898/1828] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:47.944814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:47.949740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d458457-a95f-4f08-b6fa-fb6d59d01ced service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.963390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.963390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.963390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.963390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] PATH_INFO: `/v3/projects/139c967172e441049771fe04629ca839` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:47.964159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=139c967172e441049771fe04629ca839)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:47.965634 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:47.966017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8f32-97ef-4700-973e-7446c4411724 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:47.980184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145a208-9902-413d-9c39-3a935367044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:47.981666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145a208-9902-413d-9c39-3a935367044a admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:47.982148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145a208-9902-413d-9c39-3a935367044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:47.982494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145a208-9902-413d-9c39-3a935367044a admin admin] PATH_INFO: `/v3/projects/ca4452ae20d04bb7898c23de37fa2373/users/20d7718e7d4443719ad14435dd17475b/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:48.007550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145a208-9902-413d-9c39-3a935367044a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4452ae20d04bb7898c23de37fa2373, user_id=20d7718e7d4443719ad14435dd17475b, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:48.013684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145a208-9902-413d-9c39-3a935367044a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:48.104011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 899/1829] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:47 2022] PUT /identity/v3/projects/ca4452ae20d04bb7898c23de37fa2373/users/20d7718e7d4443719ad14435dd17475b/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:48.105777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0b64-3a49-498e-ae9a-8600fd8b975d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:48.106357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0b64-3a49-498e-ae9a-8600fd8b975d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:48.107165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0b64-3a49-498e-ae9a-8600fd8b975d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:48.110028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139c967172e441049771fe04629ca839', 'request_id': 'req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:48.120921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139c967172e441049771fe04629ca839', 'request_id': 'req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:48.139818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139c967172e441049771fe04629ca839', 'request_id': 'req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:48.154671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-51bf0b64-3a49-498e-ae9a-8600fd8b975d None None] MFA Rules not processed for user `20d7718e7d4443719ad14435dd17475b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:48.154671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51bf0b64-3a49-498e-ae9a-8600fd8b975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:48.303069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 900/1830] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:48.304238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf0b64-3a49-498e-ae9a-8600fd8b975d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:48.405279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf0b64-3a49-498e-ae9a-8600fd8b975d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:48.411989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:48.412727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:48.413069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:48.413394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:48.414169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:48.416562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:48.683746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] The token cache is being invalidate because project 139c967172e441049771fe04629ca839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:48.684234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:48.684772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c6c4c351-2c1b-4d07-b7e8-3ba6054827b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:48.686374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 931/1831] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:47 2022] DELETE /identity/v3/projects/139c967172e441049771fe04629ca839 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:48.692020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab73ae4-fe7b-4d2f-92b6-847691720e56 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:48.692307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab73ae4-fe7b-4d2f-92b6-847691720e56 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:48.692579 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab73ae4-fe7b-4d2f-92b6-847691720e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:48.711612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2ab73ae4-fe7b-4d2f-92b6-847691720e56 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:48.712896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2ab73ae4-fe7b-4d2f-92b6-847691720e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:48.730440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 932/1832] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:48.752522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377deda-d854-485a-8e04-4594603210bb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:48.752522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377deda-d854-485a-8e04-4594603210bb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:48.752522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377deda-d854-485a-8e04-4594603210bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:48.775297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4377deda-d854-485a-8e04-4594603210bb None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:48.776918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4377deda-d854-485a-8e04-4594603210bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:48.790005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 933/1833] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:48.793854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377deda-d854-485a-8e04-4594603210bb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:48.804955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377deda-d854-485a-8e04-4594603210bb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:48.810052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:48.810717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:48.811071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:48.811589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:48.812352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:48.812654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:48.814274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:48.831201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 934/1834] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:48.832397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:48.850633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d16cbe-4d6b-4623-91a7-f6cb93599cd1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:48.853898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:48.854566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:48.854851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:48.855157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:48.855896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:48.857806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.149584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 935/1835] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:48 2022] POST /identity/v3/projects => generated 375 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:49.153194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 901/1836] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:48 2022] POST /identity/v3/projects => generated 381 bytes in 846 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:49.153801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.154055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.155912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:49.302607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78542390-3e61-46b8-95ec-a8156bbf96f2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.302714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd47b0-4227-4b4a-a245-b820e527812a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.306969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.307923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.308367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.308785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.309192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cc048-174f-4023-8f64-1fd4b596f959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.310919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cc048-174f-4023-8f64-1fd4b596f959 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.310919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cc048-174f-4023-8f64-1fd4b596f959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.310919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cc048-174f-4023-8f64-1fd4b596f959 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.311523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.313139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403cc048-174f-4023-8f64-1fd4b596f959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.313885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.316500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403cc048-174f-4023-8f64-1fd4b596f959 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.435312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 902/1837] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:49 2022] POST /identity/v3/users => generated 297 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:49.435312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9047a-222f-4490-94ec-b0904d4e0889 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.435312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9047a-222f-4490-94ec-b0904d4e0889 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.435312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9047a-222f-4490-94ec-b0904d4e0889 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.436849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 936/1838] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:49 2022] POST /identity/v3/users => generated 288 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:49.437987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.443552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 903/1839] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:15:49 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:15:49.444785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9047a-222f-4490-94ec-b0904d4e0889 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.452959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1832db2-af32-4d60-b4c8-c50fdf782267 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.467999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813808f-7f18-4d43-a2a0-0b909676740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.469020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813808f-7f18-4d43-a2a0-0b909676740b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.476283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813808f-7f18-4d43-a2a0-0b909676740b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.476283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813808f-7f18-4d43-a2a0-0b909676740b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.476283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813808f-7f18-4d43-a2a0-0b909676740b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.476283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813808f-7f18-4d43-a2a0-0b909676740b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.500222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9047a-222f-4490-94ec-b0904d4e0889 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.524755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 937/1840] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:49.528270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add8461-b46c-4346-b090-937f7d9f9e5c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.528270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add8461-b46c-4346-b090-937f7d9f9e5c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.528453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add8461-b46c-4346-b090-937f7d9f9e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.531017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.534840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.534840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.534840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.534840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.537317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.555422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 904/1841] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:49.557232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.558281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3add8461-b46c-4346-b090-937f7d9f9e5c None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:49.560714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3add8461-b46c-4346-b090-937f7d9f9e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:49.573593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0acfd-2a7c-4712-9d7a-e98f84c8ad2c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.584013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.585073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.585526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.585904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] PATH_INFO: `/v3/projects/5a3618d99a634df99d477cd880585902/users/e73747a1817a4924a8848c98fab7055b/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.605809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3618d99a634df99d477cd880585902, user_id=e73747a1817a4924a8848c98fab7055b, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.609633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.661938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 938/1842] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:15:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:49.663062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add8461-b46c-4346-b090-937f7d9f9e5c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.683252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add8461-b46c-4346-b090-937f7d9f9e5c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.683804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.684495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.684984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.685320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.686319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.688328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.703528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 939/1843] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:49.704496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.714785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:49.715346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 905/1844] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:49 2022] PUT /identity/v3/projects/5a3618d99a634df99d477cd880585902/users/e73747a1817a4924a8848c98fab7055b/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:49.716332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.719307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:49.726971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fb88b-e60d-4a73-834f-b5b0b8a0401a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.838499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765b42-3901-43e4-a35b-45e017cd6a31 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.840815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.841967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.842384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.842686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] PATH_INFO: `/v3/projects/b2960247ce954c1bbc611ff54b9f461c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.843968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2960247ce954c1bbc611ff54b9f461c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.845974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.852043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.852771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.853556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.853844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.864911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.866713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:49.882285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 906/1845] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:49.883117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:49.901702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be142-569b-4185-8ba8-a5c93bbde637 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:49.907704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:49.908616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:49.908958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:49.909304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] PATH_INFO: `/v3/projects/64d4f79587e14d478983e89f0f8b227b/users/ccf1fc2abb1e403e84990950c62a0088/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:49.932595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d4f79587e14d478983e89f0f8b227b, user_id=ccf1fc2abb1e403e84990950c62a0088, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:49.938237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:50.048402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 907/1846] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:49 2022] PUT /identity/v3/projects/64d4f79587e14d478983e89f0f8b227b/users/ccf1fc2abb1e403e84990950c62a0088/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:50.049206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:50.057949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2960247ce954c1bbc611ff54b9f461c', 'request_id': 'req-8508db93-3c1d-4f61-bcc5-60852ec6d068'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.064476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2960247ce954c1bbc611ff54b9f461c', 'request_id': 'req-8508db93-3c1d-4f61-bcc5-60852ec6d068'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.072849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2960247ce954c1bbc611ff54b9f461c', 'request_id': 'req-8508db93-3c1d-4f61-bcc5-60852ec6d068'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.129245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f8db-7e54-4e94-bf69-2a539ca12538 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:50.137717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92925032-37ef-4e5c-9ae4-2f6dc27cc891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:50.137717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92925032-37ef-4e5c-9ae4-2f6dc27cc891 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.137717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92925032-37ef-4e5c-9ae4-2f6dc27cc891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.137717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92925032-37ef-4e5c-9ae4-2f6dc27cc891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.138048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92925032-37ef-4e5c-9ae4-2f6dc27cc891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:50.138048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92925032-37ef-4e5c-9ae4-2f6dc27cc891 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:50.149489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 908/1847] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:50.150786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.151076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.151351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.163834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] The token cache is being invalidate because project b2960247ce954c1bbc611ff54b9f461c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:50.172142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.172142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.172142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 940/1848] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:49 2022] DELETE /identity/v3/projects/b2960247ce954c1bbc611ff54b9f461c => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:50.172142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:50.172142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:50.180320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] MFA Rules not processed for user `ccf1fc2abb1e403e84990950c62a0088`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:50.181614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:50.248218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508db93-3c1d-4f61-bcc5-60852ec6d068 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:50.254512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:50.255414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.256091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.256091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] PATH_INFO: `/v3/projects/5a3618d99a634df99d477cd880585902/users/e73747a1817a4924a8848c98fab7055b/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.258000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3618d99a634df99d477cd880585902, user_id=e73747a1817a4924a8848c98fab7055b, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:50.260777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:50.261290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 909/1849] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:50.265743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:50.267124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:50.272176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 941/1850] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:50 2022] PUT /identity/v3/projects/5a3618d99a634df99d477cd880585902/users/e73747a1817a4924a8848c98fab7055b/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:50.282019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:50.282862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a431a7-3dfc-4df5-a032-15b2e7815217 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:50.286792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:50.366289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:50.368458 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.369250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86be7-cfeb-43bf-bab9-1470572bf9d6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:50.372295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.372485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] PATH_INFO: `/v3/users/5f941f846c8448d296923c2a8d786ea7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.374562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f941f846c8448d296923c2a8d786ea7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:50.374909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b38120-9c69-494f-a953-6e7f01c124cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:50.375793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38120-9c69-494f-a953-6e7f01c124cd admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.376076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38120-9c69-494f-a953-6e7f01c124cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.376346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38120-9c69-494f-a953-6e7f01c124cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.377090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b38120-9c69-494f-a953-6e7f01c124cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:50.379296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b38120-9c69-494f-a953-6e7f01c124cd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:50.380684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:50.534013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 942/1851] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:50 2022] POST /identity/v3/projects => generated 375 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:50.534013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9483-4142-4abf-a8ca-f733075d65be None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.534013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9483-4142-4abf-a8ca-f733075d65be None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.535672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9483-4142-4abf-a8ca-f733075d65be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.598516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0ffa9483-4142-4abf-a8ca-f733075d65be None None] MFA Rules not processed for user `e73747a1817a4924a8848c98fab7055b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:50.599930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ffa9483-4142-4abf-a8ca-f733075d65be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:50.651463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f941f846c8448d296923c2a8d786ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.699114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 943/1852] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:50.700573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9483-4142-4abf-a8ca-f733075d65be None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:50.762917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f941f846c8448d296923c2a8d786ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.767318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f941f846c8448d296923c2a8d786ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.793076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f941f846c8448d296923c2a8d786ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.800991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f941f846c8448d296923c2a8d786ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:50.809129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 910/1853] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:50 2022] DELETE /identity/v3/users/5f941f846c8448d296923c2a8d786ea7 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:50.810205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:50.812884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:50.816987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9483-4142-4abf-a8ca-f733075d65be None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:50.927859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973c46a-b686-4fa6-9fee-5cde518a74a4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:50.934994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:50.935669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.936045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.936276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.941637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:50.961051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:50.961981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:50.962126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:50.962422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:50.963043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:50.965600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.017880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:51.020913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.035256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 911/1854] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:51.036727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.038764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:51.059366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 944/1855] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:50 2022] POST /identity/v3/users => generated 294 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:51.065546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.068526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:51.075171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184270f-5758-4cbd-b4b1-7edbc7908695 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:51.084543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:51.084543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.084543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.084758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] PATH_INFO: `/v3/users/02a5d450f56d41ab95da3024a68d172a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.085844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a5d450f56d41ab95da3024a68d172a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:51.088063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.116990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142185eb-05ec-48c6-8818-db45c82b61a9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:51.123795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:51.124534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.124809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.125085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.125819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:51.127418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.140332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 945/1856] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:51.144394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.155215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0515aa2-5c72-40e5-b3b1-75685f3127e3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:51.160501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:51.161495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.162265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.162265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] PATH_INFO: `/v3/projects/fe94fb65583547389eae8486cc576e36/users/96a97385a3a5495baaa7c82e3c86c0de/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.181823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe94fb65583547389eae8486cc576e36, user_id=96a97385a3a5495baaa7c82e3c86c0de, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:51.186211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.291596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 946/1857] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:51 2022] PUT /identity/v3/projects/fe94fb65583547389eae8486cc576e36/users/96a97385a3a5495baaa7c82e3c86c0de/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:15:51.299100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.347456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a5d450f56d41ab95da3024a68d172a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.432276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a5d450f56d41ab95da3024a68d172a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.433132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b4601-4c2b-4538-bc58-7d7de4f697c5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:51.433257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a5d450f56d41ab95da3024a68d172a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.439273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:51.441332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.442141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.442426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.443180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:51.444982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.452856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a5d450f56d41ab95da3024a68d172a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.459990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 947/1858] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:51.466311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a5d450f56d41ab95da3024a68d172a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.468876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.471071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:51.480505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 912/1859] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:51 2022] DELETE /identity/v3/users/02a5d450f56d41ab95da3024a68d172a => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:51.481488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.483189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:51.490878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3172e-fc57-4018-b472-968bdbba8b73 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:51.586758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bb745-1960-4440-892e-7e8ea0c785f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:51.587868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bb745-1960-4440-892e-7e8ea0c785f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.588214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bb745-1960-4440-892e-7e8ea0c785f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.588494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bb745-1960-4440-892e-7e8ea0c785f5 admin admin] PATH_INFO: `/v3/projects/fe94fb65583547389eae8486cc576e36/users/96a97385a3a5495baaa7c82e3c86c0de/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.590700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bb745-1960-4440-892e-7e8ea0c785f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe94fb65583547389eae8486cc576e36, user_id=96a97385a3a5495baaa7c82e3c86c0de, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:51.593557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bb745-1960-4440-892e-7e8ea0c785f5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.605013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5beab2-1536-481e-bbd0-a9b8943b9351 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:51.608378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 948/1860] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:51 2022] PUT /identity/v3/projects/fe94fb65583547389eae8486cc576e36/users/96a97385a3a5495baaa7c82e3c86c0de/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:51.617067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257b9b-ced9-48db-9df5-9ef250e56af0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.617361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257b9b-ced9-48db-9df5-9ef250e56af0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.617625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257b9b-ced9-48db-9df5-9ef250e56af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.636276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7c257b9b-ced9-48db-9df5-9ef250e56af0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:51.637486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7c257b9b-ced9-48db-9df5-9ef250e56af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:51.680759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:51.681633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.681913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.682183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] PATH_INFO: `/v3/projects/4ba02b377c734d36bc7f1fc8cd670ea8` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.683467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba02b377c734d36bc7f1fc8cd670ea8)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:51.685434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:51.720911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 949/1861] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:51.723184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:51.723483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:51.723787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:51.758299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] MFA Rules not processed for user `96a97385a3a5495baaa7c82e3c86c0de`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:51.760344 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:51.828028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba02b377c734d36bc7f1fc8cd670ea8', 'request_id': 'req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.875086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 950/1862] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:51.876095 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.877790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:51.923184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba02b377c734d36bc7f1fc8cd670ea8', 'request_id': 'req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.930726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba02b377c734d36bc7f1fc8cd670ea8', 'request_id': 'req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.939341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187fe88-d58e-421e-a0bd-6cae4299f8d0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:51.948983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] The token cache is being invalidate because project 4ba02b377c734d36bc7f1fc8cd670ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:51.949345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.950265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:51.958403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 913/1863] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:51 2022] DELETE /identity/v3/projects/4ba02b377c734d36bc7f1fc8cd670ea8 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:51.958403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:51.958403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:52.006509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd10419-9a26-4a63-8a50-d75662c4dffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.007100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd10419-9a26-4a63-8a50-d75662c4dffd service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.007379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd10419-9a26-4a63-8a50-d75662c4dffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.007698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd10419-9a26-4a63-8a50-d75662c4dffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.009472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fd10419-9a26-4a63-8a50-d75662c4dffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:52.047517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b321da5-b8ab-4d2d-a8b8-adbfb290eb2d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.053996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.055849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.056105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.056423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.058742 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:52.090891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd10419-9a26-4a63-8a50-d75662c4dffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.092372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd10419-9a26-4a63-8a50-d75662c4dffd service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.117944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 951/1864] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:52.120508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48d83-56a4-47f6-a039-74f6555d09be None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.120803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48d83-56a4-47f6-a039-74f6555d09be None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.121087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48d83-56a4-47f6-a039-74f6555d09be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.145270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.147202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.147335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cec48d83-56a4-47f6-a039-74f6555d09be None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:52.148535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cec48d83-56a4-47f6-a039-74f6555d09be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:52.159791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 952/1865] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:52.162924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 914/1866] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:52.166549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.170122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.170122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.200191 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:52.201841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:52.226246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 953/1867] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:52.234701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.248278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb822aa9-8996-4b72-a3fb-637c8ca7fc7f service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.255229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.256475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.256911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.257296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.258125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:52.258530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.260318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.272577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.275676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:52.282328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 915/1868] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:52.283250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.294276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05718d9a-6a33-4799-96ab-191b7672ca26 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.301139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.302061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.302437 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.303234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.303577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.306281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.385375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa245b8-0492-4bad-92be-df2f2b1fc7db None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.391794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e73747a1817a4924a8848c98fab7055b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3618d99a634df99d477cd880585902', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.392541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.392821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.393148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.394630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:52.394630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.399746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.412262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 954/1869] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:52.416716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.427127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3109f-3658-4cc3-925e-4d1c57ca0927 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.432336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e73747a1817a4924a8848c98fab7055b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3618d99a634df99d477cd880585902', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.433123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.433506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.433883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.434935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.436882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.442024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 916/1870] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:52 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:52.442530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.532820 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 955/1871] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:52 2022] POST /identity/v3/projects => generated 354 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:52.537154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.552436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b0831-0563-45f7-bfd8-bc796269ae48 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.655001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.656264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.656695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.657071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.658493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.663327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.665588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb56c7-35c3-4688-9a34-4d24434078ba tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.670846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e73747a1817a4924a8848c98fab7055b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3618d99a634df99d477cd880585902', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.671540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.672121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.672401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.673147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:52.674099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.677105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.692165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 956/1872] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:52.696460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.698436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:52.714244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39fb84-d619-458e-8ca2-972e9b92eae8 tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.719968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.720830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.721115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.721696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] PATH_INFO: `/v3/projects/3af9e1872f144599807c605bff21f770` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.724055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af9e1872f144599807c605bff21f770)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.725918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.786872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 917/1873] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:52 2022] POST /identity/v3/users => generated 284 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:52.790622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.821089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cea90-abbc-4442-8c6a-b60a6753ddc7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.843017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.844024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.848766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.849640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.852306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.854441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.860293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af9e1872f144599807c605bff21f770', 'request_id': 'req-6152d20d-3388-4fae-8800-f2bf45a05e88'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:52.872134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af9e1872f144599807c605bff21f770', 'request_id': 'req-6152d20d-3388-4fae-8800-f2bf45a05e88'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:52.876233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 918/1874] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:52.879817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:52.885523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af9e1872f144599807c605bff21f770', 'request_id': 'req-6152d20d-3388-4fae-8800-f2bf45a05e88'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:52.892462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7eb52-0e9c-4023-8d04-40b645c80091 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:52.967843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e73747a1817a4924a8848c98fab7055b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3618d99a634df99d477cd880585902', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:52.968773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:52.969063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:52.971096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] PATH_INFO: `/v3/projects/f2ebd63d496b420f9c858d8ae3547a4c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:52.971096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f2ebd63d496b420f9c858d8ae3547a4c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:52.972950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:52.994347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] The token cache is being invalidate because project 3af9e1872f144599807c605bff21f770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:52.994702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:52.995696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:52.997297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 957/1875] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:52 2022] DELETE /identity/v3/projects/3af9e1872f144599807c605bff21f770 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:53.000920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:53.002246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:53.110923 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ebd63d496b420f9c858d8ae3547a4c', 'request_id': 'req-2bdc117e-bed2-4d92-9f8c-741701a1f34d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.114668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152d20d-3388-4fae-8800-f2bf45a05e88 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:53.198958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:53.199659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:53.199944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:53.200223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:53.200943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:53.202550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:53.213379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ebd63d496b420f9c858d8ae3547a4c', 'request_id': 'req-2bdc117e-bed2-4d92-9f8c-741701a1f34d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.220162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ebd63d496b420f9c858d8ae3547a4c', 'request_id': 'req-2bdc117e-bed2-4d92-9f8c-741701a1f34d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.225463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 958/1876] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:53.226341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:53.229744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:53.239560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] The token cache is being invalidate because project f2ebd63d496b420f9c858d8ae3547a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:53.239896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.240531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.242126 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 919/1877] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:52 2022] DELETE /identity/v3/projects/f2ebd63d496b420f9c858d8ae3547a4c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:53.243089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:53.244835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:53.247132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe04f3-895a-4234-a79b-4aeea92fc154 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:53.264034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc117e-bed2-4d92-9f8c-741701a1f34d tempest-VolumeQuotaClassesTest-1240080712 tempest-VolumeQuotaClassesTest-1240080712-project-admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:53.264325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:53.265224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:53.265516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:53.265793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] PATH_INFO: `/v3/users/3fba0a4a385940e9a58777acfd73cbb2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:53.269426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fba0a4a385940e9a58777acfd73cbb2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:53.271409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c3171-6324-4795-a4ac-0c69b39d7bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:53.271537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:53.272590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3171-6324-4795-a4ac-0c69b39d7bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:53.272895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3171-6324-4795-a4ac-0c69b39d7bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:53.273140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3171-6324-4795-a4ac-0c69b39d7bbd admin admin] PATH_INFO: `/v3/projects/1176d818e9c14a29ba6bd2b38539f8c2/users/b2acec2e0dd24d32823490d37ec34850/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:53.298615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c3171-6324-4795-a4ac-0c69b39d7bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1176d818e9c14a29ba6bd2b38539f8c2, user_id=b2acec2e0dd24d32823490d37ec34850, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:53.303742 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c3171-6324-4795-a4ac-0c69b39d7bbd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:53.395362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 920/1878] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:53 2022] PUT /identity/v3/projects/1176d818e9c14a29ba6bd2b38539f8c2/users/b2acec2e0dd24d32823490d37ec34850/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:53.406544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0257-e386-4b07-9503-47a4cff2b585 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:53.406837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0257-e386-4b07-9503-47a4cff2b585 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:53.407155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0257-e386-4b07-9503-47a4cff2b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:53.441753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-145c0257-e386-4b07-9503-47a4cff2b585 None None] MFA Rules not processed for user `b2acec2e0dd24d32823490d37ec34850`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:53.443019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-145c0257-e386-4b07-9503-47a4cff2b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:53.502512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fba0a4a385940e9a58777acfd73cbb2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.531893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 921/1879] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:53.536479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0257-e386-4b07-9503-47a4cff2b585 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:53.600547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0257-e386-4b07-9503-47a4cff2b585 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:53.605056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:53.605690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:53.605970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:53.606242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:53.606919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:53.608849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:53.614805 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fba0a4a385940e9a58777acfd73cbb2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.622747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fba0a4a385940e9a58777acfd73cbb2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.623415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fba0a4a385940e9a58777acfd73cbb2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.634998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fba0a4a385940e9a58777acfd73cbb2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:53.640754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 959/1880] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:53 2022] DELETE /identity/v3/users/3fba0a4a385940e9a58777acfd73cbb2 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:53.731488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 922/1881] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:53 2022] POST /identity/v3/projects => generated 369 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:53.735547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:53.737212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:53.816351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d675c3-b1ac-47fe-8483-a2c6dd1de815 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:53.883364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:53.884091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:53.884375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:53.884816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:53.885521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:53.887453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:53.946630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:53.948449 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:53.962106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c5f3e-622d-4981-8e4d-83ed0968fcc0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:53.966730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:53.967620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:53.967924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:53.968201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] PATH_INFO: `/v3/users/20d7718e7d4443719ad14435dd17475b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:53.969365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d7718e7d4443719ad14435dd17475b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:53.971297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:54.015945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 960/1882] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:53 2022] POST /identity/v3/users => generated 291 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:54.016740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:54.029358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfcfcc-8429-4318-a358-c97c4c741bc1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:54.039039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:54.039039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.040177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.040177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.041188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:54.044320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:54.062949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 961/1883] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:54.063244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:54.065355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:54.079714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de062f-8c6f-4b5d-8ffb-9c6f4d9730c0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:54.085995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:54.086888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.087220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.087499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] PATH_INFO: `/v3/projects/0560e49f7b694d7998fe6df0d7ecc358` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.088764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0560e49f7b694d7998fe6df0d7ecc358)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:54.090751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:54.215700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d7718e7d4443719ad14435dd17475b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.242082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0560e49f7b694d7998fe6df0d7ecc358', 'request_id': 'req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.248167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0560e49f7b694d7998fe6df0d7ecc358', 'request_id': 'req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.263616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0560e49f7b694d7998fe6df0d7ecc358', 'request_id': 'req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.300864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d7718e7d4443719ad14435dd17475b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.301552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d7718e7d4443719ad14435dd17475b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.316540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d7718e7d4443719ad14435dd17475b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.327178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d7718e7d4443719ad14435dd17475b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.349392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 923/1884] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:53 2022] DELETE /identity/v3/users/20d7718e7d4443719ad14435dd17475b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:15:54.350968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:54.352771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:54.416808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] The token cache is being invalidate because project 0560e49f7b694d7998fe6df0d7ecc358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:54.417172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.419195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e45a6c63-1f77-4bf7-bddf-7e0ffcecdc26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:54.423528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 962/1885] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:54 2022] DELETE /identity/v3/projects/0560e49f7b694d7998fe6df0d7ecc358 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:54.440333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9716c1-20ae-40c1-b195-71db79b93d44 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.441248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9716c1-20ae-40c1-b195-71db79b93d44 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.441736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9716c1-20ae-40c1-b195-71db79b93d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.504571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fe9716c1-20ae-40c1-b195-71db79b93d44 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:54.509130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fe9716c1-20ae-40c1-b195-71db79b93d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:54.551194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 963/1886] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:54.556556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886602e-65f1-4b5a-b4fd-6795f0938149 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:54.557680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.558851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.562385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.574097 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54656d-f181-4c91-8751-b4ef7b193059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:54.576949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54656d-f181-4c91-8751-b4ef7b193059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.576949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54656d-f181-4c91-8751-b4ef7b193059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.576949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54656d-f181-4c91-8751-b4ef7b193059 admin admin] PATH_INFO: `/v3/projects/ef85ed99f5e44d1489e690354654aff3/users/83aae14cc00a44e7a548dcc2b71ea1ec/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.609235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:54.610084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54656d-f181-4c91-8751-b4ef7b193059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef85ed99f5e44d1489e690354654aff3, user_id=83aae14cc00a44e7a548dcc2b71ea1ec, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:54.611528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:54.617826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54656d-f181-4c91-8751-b4ef7b193059 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:54.650212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 964/1887] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:54.651207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:54.652836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:54.730830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 924/1888] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:54 2022] PUT /identity/v3/projects/ef85ed99f5e44d1489e690354654aff3/users/83aae14cc00a44e7a548dcc2b71ea1ec/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:54.737727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.738711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.739233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.741819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e30c6c-41e5-428e-a0b1-aa4013e42cb5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:54.764762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:54.766085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:54.825110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:54.826417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.826862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.827255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.891969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 925/1889] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:54.897964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:54.902324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:54.935431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404bc2d-a556-457d-a18a-a9ba4da70bf0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:54.943925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:54.945134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.945757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.946751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.948273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:54.952462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:54.971358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 926/1890] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:54.971358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:54.977921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:54.979611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:54.982145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a857-a6f7-4005-bea4-37ecfcddd5fd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:54.988678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:54.989419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:54.989812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:54.998560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:54.998560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:54.998560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:54.998560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:54.999218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 965/1891] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:55.006898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 927/1892] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:15:55.008255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.011327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.042027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11834a6-dc24-470d-a1fd-63c5d241b483 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.053716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f54c-8c46-4380-bf64-e5c7465ca328 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.060004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.060004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.060004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.060004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.061179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.061521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.079019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382f758-bb8f-414b-a013-639d53971031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.080156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f758-bb8f-414b-a013-639d53971031 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.080680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f758-bb8f-414b-a013-639d53971031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.081185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f758-bb8f-414b-a013-639d53971031 admin admin] PATH_INFO: `/v3/projects/ef85ed99f5e44d1489e690354654aff3/users/83aae14cc00a44e7a548dcc2b71ea1ec/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.084165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382f758-bb8f-414b-a013-639d53971031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef85ed99f5e44d1489e690354654aff3, user_id=83aae14cc00a44e7a548dcc2b71ea1ec, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.087099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382f758-bb8f-414b-a013-639d53971031 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.103166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 966/1893] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:55 2022] PUT /identity/v3/projects/ef85ed99f5e44d1489e690354654aff3/users/83aae14cc00a44e7a548dcc2b71ea1ec/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:55.117270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.117860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.118772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.150094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] MFA Rules not processed for user `83aae14cc00a44e7a548dcc2b71ea1ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:55.151312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:55.216428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 928/1894] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:55 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:55.230244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.257718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 967/1895] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:55.273576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.276040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:55.319663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192c104-0e56-4ab4-b34a-7df4ba46f71f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.323807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.324521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.324857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.325469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.325762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.328089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.349689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c99f8-19b4-479d-9da1-e7e55f88b4b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.414342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.415300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.415459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.415901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.453822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 929/1896] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:55 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:55.455238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.466264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946e704-493f-4e0c-8d22-a6510b31ee7e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.471754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.480027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.480027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.480027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.480027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.480027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.490508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 930/1897] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:55.492165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.493655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.494562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.507473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ac668-6c51-4f57-85ec-4245a4c6902b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.516294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 968/1898] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:55 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:55.516543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.518041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.518569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.519225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.520154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.523124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.539194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 931/1899] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:55.539338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.540684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:55.543496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.555642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dad42-8860-402a-b597-fc2fe27a7c04 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.561141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.562178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.562536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.562878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] PATH_INFO: `/v3/projects/0b9cf2a3d7894f1f8550e35bf6787ba6/users/0557244527f04a0180fe7d82a8232487/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.570559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369831ec-c815-4e2c-a64a-3256c4034838 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.575928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.577006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.577337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.577665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] PATH_INFO: `/v3/projects/ca4452ae20d04bb7898c23de37fa2373` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.579032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4452ae20d04bb7898c23de37fa2373)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.581462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.583245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9cf2a3d7894f1f8550e35bf6787ba6, user_id=0557244527f04a0180fe7d82a8232487, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.586308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.664818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 932/1900] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:55 2022] PUT /identity/v3/projects/0b9cf2a3d7894f1f8550e35bf6787ba6/users/0557244527f04a0180fe7d82a8232487/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:55.669994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.671619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:55.731819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4452ae20d04bb7898c23de37fa2373', 'request_id': 'req-907a236d-b5ed-4803-8300-3bdd65f63ccd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:55.734677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6bd4a-ce21-4706-b094-7c7853306bc6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.738310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4452ae20d04bb7898c23de37fa2373', 'request_id': 'req-907a236d-b5ed-4803-8300-3bdd65f63ccd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:55.745424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4452ae20d04bb7898c23de37fa2373', 'request_id': 'req-907a236d-b5ed-4803-8300-3bdd65f63ccd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:55.795489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.796407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.796690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.796965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] PATH_INFO: `/v3/users/ccf1fc2abb1e403e84990950c62a0088` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.798233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf1fc2abb1e403e84990950c62a0088)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.800866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:55.828140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] The token cache is being invalidate because project ca4452ae20d04bb7898c23de37fa2373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:55.828369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:55.829005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-907a236d-b5ed-4803-8300-3bdd65f63ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:55.830613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 969/1901] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:55 2022] DELETE /identity/v3/projects/ca4452ae20d04bb7898c23de37fa2373 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:15:55.831996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.832281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.832557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.859677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] MFA Rules not processed for user `0557244527f04a0180fe7d82a8232487`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:55.860931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:55.931396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 970/1902] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:55.932276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:55.933551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:55.946668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c95ca-bbf6-46d7-be9b-4b555f0ad945 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:55.951389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:55.952313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:55.952575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:55.952855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] PATH_INFO: `/v3/users/e73747a1817a4924a8848c98fab7055b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:55.954059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73747a1817a4924a8848c98fab7055b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:55.956088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:56.003249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf1fc2abb1e403e84990950c62a0088'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.148981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf1fc2abb1e403e84990950c62a0088'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.149648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf1fc2abb1e403e84990950c62a0088'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.165366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf1fc2abb1e403e84990950c62a0088'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.168741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73747a1817a4924a8848c98fab7055b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.173693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf1fc2abb1e403e84990950c62a0088'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.179671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 933/1903] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:55 2022] DELETE /identity/v3/users/ccf1fc2abb1e403e84990950c62a0088 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:56.180669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:56.182498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:56.252689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326b1d4-bb57-4bfc-b628-321520cde82a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:56.258857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:56.259602 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:56.259993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:56.260342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:56.261517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:56.263115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:56.266214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73747a1817a4924a8848c98fab7055b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.276400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73747a1817a4924a8848c98fab7055b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.280574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73747a1817a4924a8848c98fab7055b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.301327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ae63e317-f8cc-442e-8ea2-cb00fee07e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73747a1817a4924a8848c98fab7055b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:56.308079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 971/1904] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:55 2022] DELETE /identity/v3/users/e73747a1817a4924a8848c98fab7055b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:56.309656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:56.310048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:56.310428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:56.333621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:56.338235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:56.364685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 934/1905] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:56 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:56.369489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:56.372711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:56.474349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 972/1906] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:56.486324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:56.488589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:56.499769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9b17f-792a-4fa1-8a8f-b85883cc87a1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:56.505346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:56.506015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:56.506292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:56.506562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:56.507249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:56.509390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:56.548138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2249e28-c96c-492a-a7cd-38d053ec054d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:56.610172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:56.611225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:56.611889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:56.613027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:56.627376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:56.629303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:56.644155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 935/1907] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:56 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:56.648542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:56.663673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 973/1908] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:15:56.670777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc7c0-2124-4c8d-baf6-64a32e940b8d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:56.676360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:56.677834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:56.678847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:56.679133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:56.679977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:56.681713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:56.692605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 936/1909] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:56.696854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:56.710119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2bac-d341-4874-918a-c76c009a197d service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:56.716817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:56.719683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:56.719683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:56.719683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] PATH_INFO: `/v3/projects/2c07684217a64763a036cb4652806dcc/users/57312d73a1574184a15f6bd3d56dbfff/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:56.755322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c07684217a64763a036cb4652806dcc, user_id=57312d73a1574184a15f6bd3d56dbfff, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:56.756960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:56.843714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:56.847803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:56.861001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db55b4-1932-43b2-a8dc-71df03bc9d37 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:56.866431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:56.867303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:56.867593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:56.867897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] PATH_INFO: `/v3/projects/5a3618d99a634df99d477cd880585902` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:56.871684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3618d99a634df99d477cd880585902)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:56.871684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:56.887022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 974/1910] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:56 2022] PUT /identity/v3/projects/2c07684217a64763a036cb4652806dcc/users/57312d73a1574184a15f6bd3d56dbfff/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:15:56.894086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.029339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3618d99a634df99d477cd880585902', 'request_id': 'req-52f33bdf-4098-4ea6-8fe8-2649d7de6679'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.030867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76341a2b-4762-4037-9657-cc95ef71f9c8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.121527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.122381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.122907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.123351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.124230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.126626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.130269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3618d99a634df99d477cd880585902', 'request_id': 'req-52f33bdf-4098-4ea6-8fe8-2649d7de6679'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.152003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3618d99a634df99d477cd880585902', 'request_id': 'req-52f33bdf-4098-4ea6-8fe8-2649d7de6679'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.152189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 975/1911] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:15:57.160993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.162485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] The token cache is being invalidate because project 5a3618d99a634df99d477cd880585902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:57.162862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.164483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52f33bdf-4098-4ea6-8fe8-2649d7de6679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.168459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 937/1912] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:56 2022] DELETE /identity/v3/projects/5a3618d99a634df99d477cd880585902 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:57.174046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.174506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.174990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.175450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e006d4c-8485-4c57-80a9-104013754531 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.180999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aa30f-c83a-42b9-9d7a-70ecbb3674f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.185482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa30f-c83a-42b9-9d7a-70ecbb3674f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.185482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa30f-c83a-42b9-9d7a-70ecbb3674f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.185482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa30f-c83a-42b9-9d7a-70ecbb3674f5 admin admin] PATH_INFO: `/v3/projects/2c07684217a64763a036cb4652806dcc/users/57312d73a1574184a15f6bd3d56dbfff/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.185482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aa30f-c83a-42b9-9d7a-70ecbb3674f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c07684217a64763a036cb4652806dcc, user_id=57312d73a1574184a15f6bd3d56dbfff, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.187218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aa30f-c83a-42b9-9d7a-70ecbb3674f5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.195419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:57.198300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.199884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 976/1913] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:57 2022] PUT /identity/v3/projects/2c07684217a64763a036cb4652806dcc/users/57312d73a1574184a15f6bd3d56dbfff/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:15:57.213263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69cc0-a3ca-4b67-8306-147dcdf8ba6f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.220810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69cc0-a3ca-4b67-8306-147dcdf8ba6f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.220810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69cc0-a3ca-4b67-8306-147dcdf8ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.248577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-aba69cc0-a3ca-4b67-8306-147dcdf8ba6f None None] MFA Rules not processed for user `57312d73a1574184a15f6bd3d56dbfff`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:57.253106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aba69cc0-a3ca-4b67-8306-147dcdf8ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.283603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 938/1914] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:57.290091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.291176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.304000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef08406-3e22-4a12-a808-233ad52c4f8b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.309619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.310606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.311529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.311529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] PATH_INFO: `/v3/projects/64d4f79587e14d478983e89f0f8b227b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.312522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d4f79587e14d478983e89f0f8b227b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.314619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.344872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 977/1915] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:57.346275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d12b5-43dc-44d4-8e76-e320545b72fa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.346557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d12b5-43dc-44d4-8e76-e320545b72fa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.347650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d12b5-43dc-44d4-8e76-e320545b72fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.365915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-527d12b5-43dc-44d4-8e76-e320545b72fa None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:57.367160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-527d12b5-43dc-44d4-8e76-e320545b72fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.375283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 978/1916] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:57.379770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d12b5-43dc-44d4-8e76-e320545b72fa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.388763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d12b5-43dc-44d4-8e76-e320545b72fa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.393558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.394639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.395147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.395577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.396357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:57.396682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.398537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.411124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 979/1917] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:57.415448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.425810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc09bfb-b542-4b30-a9e7-41b3d1ca9fe7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.431622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.432284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.432571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.432851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.433538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.435433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.462116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d4f79587e14d478983e89f0f8b227b', 'request_id': 'req-254fca21-562e-4e6b-83da-a5426fc4921d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.543065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 980/1918] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:57 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:57.543065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.545668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.562657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d4f79587e14d478983e89f0f8b227b', 'request_id': 'req-254fca21-562e-4e6b-83da-a5426fc4921d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.572062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d4f79587e14d478983e89f0f8b227b', 'request_id': 'req-254fca21-562e-4e6b-83da-a5426fc4921d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.588666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] The token cache is being invalidate because project 64d4f79587e14d478983e89f0f8b227b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:57.589570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.590583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:57.592881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 939/1919] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:57 2022] DELETE /identity/v3/projects/64d4f79587e14d478983e89f0f8b227b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:57.594635 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.597198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.631265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde4609-513b-44d4-a9f7-7827ff2ea882 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.635291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.636085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.636568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.636831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.638817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.706957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fca21-562e-4e6b-83da-a5426fc4921d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.741439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.743012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.759761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 981/1920] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:57.761264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.762701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.821042 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.821836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.822219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.823957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.824780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.826891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.853237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9c00d-837b-4351-8bec-c1cb1bc27544 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.858605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.859485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.859813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.862150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] PATH_INFO: `/v3/users/96a97385a3a5495baaa7c82e3c86c0de` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.862150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a97385a3a5495baaa7c82e3c86c0de)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:57.863809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:57.941627 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 940/1921] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:57 2022] POST /identity/v3/users => generated 294 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:57.941627 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:57.944495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:57.958851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4fe4d-dba2-4323-95fe-2b2d78d56b65 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:57.964037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:57.964520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:57.965050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:57.965277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:57.967378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:58.039533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.041152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.047763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a97385a3a5495baaa7c82e3c86c0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:58.052566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 941/1922] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:15:58.053318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.063831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cbe78-3694-4e30-abeb-9976b35db93c service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.069615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.070306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.070645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.071435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.071664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.073245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.088079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 942/1923] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:58.096490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.107503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364ab72-c329-4bca-8fe8-9458eca1fdad admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.112781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.116039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.116039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.116039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.116039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.117558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.131669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 943/1924] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:58.133134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.143292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a97385a3a5495baaa7c82e3c86c0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:58.153533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a97385a3a5495baaa7c82e3c86c0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:58.154359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5b3c-009e-4c93-919f-63f68e4706e7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.154809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a97385a3a5495baaa7c82e3c86c0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:58.160879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.162154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.162611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.163200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] PATH_INFO: `/v3/projects/3f6a70620e7b452b8fdecd66348694d2/users/2b3b3ae884074afb9ad8f62467b5de06/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.181047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ce55499c-cec2-4bed-868b-c24431a89497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a97385a3a5495baaa7c82e3c86c0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:58.187768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6a70620e7b452b8fdecd66348694d2, user_id=2b3b3ae884074afb9ad8f62467b5de06, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.191581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.192042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 982/1925] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:57 2022] DELETE /identity/v3/users/96a97385a3a5495baaa7c82e3c86c0de => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:58.262505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 944/1926] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:58 2022] PUT /identity/v3/projects/3f6a70620e7b452b8fdecd66348694d2/users/2b3b3ae884074afb9ad8f62467b5de06/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:15:58.273266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867dc4f-2adf-473f-9f8a-52053b471f76 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.273562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867dc4f-2adf-473f-9f8a-52053b471f76 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.273845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867dc4f-2adf-473f-9f8a-52053b471f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.301504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d867dc4f-2adf-473f-9f8a-52053b471f76 None None] MFA Rules not processed for user `2b3b3ae884074afb9ad8f62467b5de06`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:58.302884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d867dc4f-2adf-473f-9f8a-52053b471f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:58.371568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 983/1927] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:58.377301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.380094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:58.466783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b956d2-787e-46f2-8407-364111cc12c9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.472094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.473175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.473282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.473659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.474596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.476641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.561714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 945/1928] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:58 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:58.567030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867dc4f-2adf-473f-9f8a-52053b471f76 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.580883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867dc4f-2adf-473f-9f8a-52053b471f76 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.646494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.647345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.647621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.647916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.648595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.650507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.658914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.661889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:58.675621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc64eb0-1ae8-479a-a2e3-fa49a0aee42f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.680741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.681608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.682047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.682509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] PATH_INFO: `/v3/projects/fe94fb65583547389eae8486cc576e36` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.684075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe94fb65583547389eae8486cc576e36)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.686404 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.787873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 984/1929] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:58 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:58.791818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.809737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c67321-d521-40c5-83f8-5499f49c7905 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.817598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.825645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.825645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.825645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.825645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.825645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.836845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 985/1930] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:58.841243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.860515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75231024-a357-4868-9b95-df5c28a5f809 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.896660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe94fb65583547389eae8486cc576e36', 'request_id': 'req-92809647-abf6-4339-9317-7015bc1dfc86'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:58.952024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.952024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.952024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.952024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:58.952381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:58.952381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:58.963881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 986/1931] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:58.967714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:58.979722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d623e2-c5c0-43a7-bc96-4dfba5457631 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:58.991488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20fa95-8489-410b-9712-655e24cbe367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:58.992537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20fa95-8489-410b-9712-655e24cbe367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:58.992907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20fa95-8489-410b-9712-655e24cbe367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:58.993233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20fa95-8489-410b-9712-655e24cbe367 admin admin] PATH_INFO: `/v3/projects/4b9e8702bb9a4f108d6aeeca485b8b69/users/ca93062b9ba843adb714ecfa1fb8b7cc/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.000371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe94fb65583547389eae8486cc576e36', 'request_id': 'req-92809647-abf6-4339-9317-7015bc1dfc86'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.006530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe94fb65583547389eae8486cc576e36', 'request_id': 'req-92809647-abf6-4339-9317-7015bc1dfc86'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.008780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20fa95-8489-410b-9712-655e24cbe367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9e8702bb9a4f108d6aeeca485b8b69, user_id=ca93062b9ba843adb714ecfa1fb8b7cc, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.011452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20fa95-8489-410b-9712-655e24cbe367 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.018276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] The token cache is being invalidate because project fe94fb65583547389eae8486cc576e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:15:59.018602 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.019257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92809647-abf6-4339-9317-7015bc1dfc86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.020849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 946/1932] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:15:58 2022] DELETE /identity/v3/projects/fe94fb65583547389eae8486cc576e36 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:59.025793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1ea4-d465-47c9-ac41-a406784a9c36 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.026145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1ea4-d465-47c9-ac41-a406784a9c36 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.026521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1ea4-d465-47c9-ac41-a406784a9c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.044007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6b4b1ea4-d465-47c9-ac41-a406784a9c36 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:59.045142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6b4b1ea4-d465-47c9-ac41-a406784a9c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:59.054539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 947/1933] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:15:59.062432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a72c-f72f-4dd8-8f70-ef3d9698ab44 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.062720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a72c-f72f-4dd8-8f70-ef3d9698ab44 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.062995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a72c-f72f-4dd8-8f70-ef3d9698ab44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.082889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1ea1a72c-f72f-4dd8-8f70-ef3d9698ab44 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:59.084119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1ea1a72c-f72f-4dd8-8f70-ef3d9698ab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:59.091993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 948/1934] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:59.096176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1a72c-f72f-4dd8-8f70-ef3d9698ab44 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.106664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1a72c-f72f-4dd8-8f70-ef3d9698ab44 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.110433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.111119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.111398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.111702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.114745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:15:59.114745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.115341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.128013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 949/1935] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:15:59.132927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.146169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d19474-1af8-4ebb-b98e-3102d4cebebb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.150261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 987/1936] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:58 2022] PUT /identity/v3/projects/4b9e8702bb9a4f108d6aeeca485b8b69/users/ca93062b9ba843adb714ecfa1fb8b7cc/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:59.163994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.168011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.168011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.209136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] MFA Rules not processed for user `ca93062b9ba843adb714ecfa1fb8b7cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:15:59.210948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:59.235644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.236323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.236608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.236933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.239179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.241258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.283967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 988/1937] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:15:59.288384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.289764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:59.304513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ee3d7-35c3-4121-9047-aa3dc216475b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.309258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.309952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.310233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.310505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.311196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.313122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.331333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 950/1938] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:59.336070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.395723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455816-4dc3-440d-9720-417f47eae84c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.406089 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 989/1939] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:59.406376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.407130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.407367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.407500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.407692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.408519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.410819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.490363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d75103-e40a-4169-aef7-0d405cc50567 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.494979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.495736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.496017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.496387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.497081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.499005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.504485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 951/1940] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:15:59.504485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.508645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:59.522365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caac62-2dca-4d78-b2f5-93adf54dca01 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.527224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.528186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.528473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.528752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] PATH_INFO: `/v3/users/b2acec2e0dd24d32823490d37ec34850` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.529939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2acec2e0dd24d32823490d37ec34850)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.531883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.599795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 990/1941] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:15:59.600862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.611486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0eb915-7feb-4786-9a38-b39df14cae38 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.617102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.617853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.618198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.618536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.619319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.620953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.630846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 991/1942] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:59.631818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.641492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee200ac-e4c0-4f63-82be-7db7afb4b80f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.646283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.647184 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.647465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.647779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.648487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.650056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.661774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 992/1943] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:59.662669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.672022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2adbb-2816-495c-bb9b-9da1e03a4d63 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.678525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.679168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.679449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.679746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.680456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.682058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.692669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 993/1944] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:59.693410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.703812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d33c2-7b09-407e-b2a9-bc0acc201763 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.708254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.709119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.709400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.709677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] PATH_INFO: `/v3/projects/9848d27b0c3f4d02b827cce6050866ca/users/d8ef9790b34f406f83d6d782920e7544/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.726106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9848d27b0c3f4d02b827cce6050866ca, user_id=d8ef9790b34f406f83d6d782920e7544, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.728809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.750789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2acec2e0dd24d32823490d37ec34850'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.837678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 994/1945] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:59 2022] PUT /identity/v3/projects/9848d27b0c3f4d02b827cce6050866ca/users/d8ef9790b34f406f83d6d782920e7544/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:59.840553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.860265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2acec2e0dd24d32823490d37ec34850'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.860989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2acec2e0dd24d32823490d37ec34850'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.875399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2acec2e0dd24d32823490d37ec34850'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.882900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2acec2e0dd24d32823490d37ec34850'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:15:59.889128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 952/1946] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:15:59 2022] DELETE /identity/v3/users/b2acec2e0dd24d32823490d37ec34850 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:15:59.892772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:15:59.897873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:15:59.915020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c82a-84a8-414b-8074-9656fb4b7e1a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.973808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c7bf6-527d-453f-bc40-26465db6c324 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:15:59.980816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77229e09-358c-41a7-8a24-e03ef9acd7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.981727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77229e09-358c-41a7-8a24-e03ef9acd7ac admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.982012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77229e09-358c-41a7-8a24-e03ef9acd7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.982291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77229e09-358c-41a7-8a24-e03ef9acd7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.983201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77229e09-358c-41a7-8a24-e03ef9acd7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:15:59.983910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d37b73-2b68-4c2c-a87e-e211e1fbd09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:15:59.984923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d37b73-2b68-4c2c-a87e-e211e1fbd09b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.985034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77229e09-358c-41a7-8a24-e03ef9acd7ac admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:15:59.985130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d37b73-2b68-4c2c-a87e-e211e1fbd09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.985305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d37b73-2b68-4c2c-a87e-e211e1fbd09b admin admin] PATH_INFO: `/v3/projects/512f2c902c654000bf31972fcd402137/users/11f5dba4ab4140b18d06c1e6bd86d5ea/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:15:59.996260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 953/1947] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:15:59.998020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed5dc-1bb6-4b61-aec4-67c4a17a3cf4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:15:59.998407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed5dc-1bb6-4b61-aec4-67c4a17a3cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:15:59.998945 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aed5dc-1bb6-4b61-aec4-67c4a17a3cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.014409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d37b73-2b68-4c2c-a87e-e211e1fbd09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=512f2c902c654000bf31972fcd402137, user_id=11f5dba4ab4140b18d06c1e6bd86d5ea, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.017580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d37b73-2b68-4c2c-a87e-e211e1fbd09b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.029558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-79aed5dc-1bb6-4b61-aec4-67c4a17a3cf4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:00.031098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-79aed5dc-1bb6-4b61-aec4-67c4a17a3cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:00.040647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 954/1948] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:00.041313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aed5dc-1bb6-4b61-aec4-67c4a17a3cf4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.052928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aed5dc-1bb6-4b61-aec4-67c4a17a3cf4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.057930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.058825 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.059108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.059381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] PATH_INFO: `/v3/projects/9848d27b0c3f4d02b827cce6050866ca/users/d8ef9790b34f406f83d6d782920e7544/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.061498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9848d27b0c3f4d02b827cce6050866ca, user_id=d8ef9790b34f406f83d6d782920e7544, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.064211 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.075250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 955/1949] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:00 2022] PUT /identity/v3/projects/9848d27b0c3f4d02b827cce6050866ca/users/d8ef9790b34f406f83d6d782920e7544/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:00.075928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.081970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:00.115362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 995/1950] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:15:59 2022] PUT /identity/v3/projects/512f2c902c654000bf31972fcd402137/users/11f5dba4ab4140b18d06c1e6bd86d5ea/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:00.116766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f81d-9184-44e6-b3ea-af6b4ea6f836 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.119328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f81d-9184-44e6-b3ea-af6b4ea6f836 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.119328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f81d-9184-44e6-b3ea-af6b4ea6f836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.146655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540bc6a-b8e4-4fdf-bf98-e2d51e2905cc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.152068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3803f81d-9184-44e6-b3ea-af6b4ea6f836 None None] MFA Rules not processed for user `d8ef9790b34f406f83d6d782920e7544`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:00.153367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3803f81d-9184-44e6-b3ea-af6b4ea6f836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:00.202920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.203653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.203947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.204177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.229828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 996/1951] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:00.231544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4e5e9-1651-41f9-94e1-f720387b5fc3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.231993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4e5e9-1651-41f9-94e1-f720387b5fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.232523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4e5e9-1651-41f9-94e1-f720387b5fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.264263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bfe4e5e9-1651-41f9-94e1-f720387b5fc3 None None] MFA Rules not processed for user `11f5dba4ab4140b18d06c1e6bd86d5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:00.266739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bfe4e5e9-1651-41f9-94e1-f720387b5fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:00.285066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.286670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.298023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 956/1952] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:00.299499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.301490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:00.375972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db299c6-a1ee-481c-a6da-951e22c44159 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.384177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.384992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.385376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.385777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.401163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.407829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.415077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 957/1953] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:00.421279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 997/1954] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:00.425680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.430784 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:00.449259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32cea-0512-44dd-b21c-83a295001cc7 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.455181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.456554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.456842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.457065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.457755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.459697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.577373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 958/1955] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:00 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:00.581893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4e5e9-1651-41f9-94e1-f720387b5fc3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.594169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4e5e9-1651-41f9-94e1-f720387b5fc3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.661259 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.662146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.662429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.662899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.663616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.665617 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.819550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 998/1956] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:00 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:00.821117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.834424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61229076-2124-4bbb-b1f8-e682471346c7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.848304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.849319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.849508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.849790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.851084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.853192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.867303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 959/1957] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:00.874456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.888383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f88095-7d4f-4465-bad3-b92089c5d0a1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.894810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea1575-77dc-413e-a0ba-c22565b475c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.895498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea1575-77dc-413e-a0ba-c22565b475c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.895812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea1575-77dc-413e-a0ba-c22565b475c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.896092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea1575-77dc-413e-a0ba-c22565b475c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.897489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea1575-77dc-413e-a0ba-c22565b475c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.899226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea1575-77dc-413e-a0ba-c22565b475c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:00.910638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 999/1958] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:00.915319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:00.933779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ebb02-b630-427d-982d-629585ba8acd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:00.939465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:00.940699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:00.941077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:00.941445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] PATH_INFO: `/v3/projects/78052f5b06774f39abc51a5d70324ae4/users/cd49030cbb5943e1b31b27c10f753d5d/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:00.957976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=78052f5b06774f39abc51a5d70324ae4, user_id=cd49030cbb5943e1b31b27c10f753d5d, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:00.960981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:01.059595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 960/1959] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:00 2022] PUT /identity/v3/projects/78052f5b06774f39abc51a5d70324ae4/users/cd49030cbb5943e1b31b27c10f753d5d/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:01.075822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71945248-a786-486d-9bb8-8005f3346396 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:01.075822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71945248-a786-486d-9bb8-8005f3346396 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:01.075822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71945248-a786-486d-9bb8-8005f3346396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:01.112233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-71945248-a786-486d-9bb8-8005f3346396 None None] MFA Rules not processed for user `cd49030cbb5943e1b31b27c10f753d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:01.114141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-71945248-a786-486d-9bb8-8005f3346396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:01.211113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1000/1960] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:01.216135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:01.296032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158eadd-0c79-440f-91a9-b8f11fae44fd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:01.301836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:01.302524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:01.302815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:01.303092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:01.304038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:01.306022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:01.344596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71945248-a786-486d-9bb8-8005f3346396 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:01.346064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-71945248-a786-486d-9bb8-8005f3346396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:01.360940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71945248-a786-486d-9bb8-8005f3346396 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:01.367256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:01.368099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:01.368381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:01.368653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] PATH_INFO: `/v3/projects/1176d818e9c14a29ba6bd2b38539f8c2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:01.369897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1176d818e9c14a29ba6bd2b38539f8c2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:01.371850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:01.413671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 961/1961] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:01 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:01.414660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:01.416196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:01.483050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae241ed-4473-48ab-955c-31e6f97aa01d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:01.487757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:01.488669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:01.488950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:01.489225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] PATH_INFO: `/v3/users/0557244527f04a0180fe7d82a8232487` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:01.490436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=0557244527f04a0180fe7d82a8232487)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:01.492406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:01.507971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1176d818e9c14a29ba6bd2b38539f8c2', 'request_id': 'req-68731b98-27b4-40d7-bd83-fd24c62ec703'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.513298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1176d818e9c14a29ba6bd2b38539f8c2', 'request_id': 'req-68731b98-27b4-40d7-bd83-fd24c62ec703'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.520619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1176d818e9c14a29ba6bd2b38539f8c2', 'request_id': 'req-68731b98-27b4-40d7-bd83-fd24c62ec703'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.628203 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] The token cache is being invalidate because project 1176d818e9c14a29ba6bd2b38539f8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:01.628729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.629513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.632674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1001/1962] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:01 2022] DELETE /identity/v3/projects/1176d818e9c14a29ba6bd2b38539f8c2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:01.632674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:01.634024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:01.724771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0557244527f04a0180fe7d82a8232487'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.753806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68731b98-27b4-40d7-bd83-fd24c62ec703 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:01.759780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:01.760468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:01.760747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:01.761022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:01.761720 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:01.763697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:01.782696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0557244527f04a0180fe7d82a8232487'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.783299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0557244527f04a0180fe7d82a8232487'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.797028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0557244527f04a0180fe7d82a8232487'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.804348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0557244527f04a0180fe7d82a8232487'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:01.810570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 962/1963] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:01 2022] DELETE /identity/v3/users/0557244527f04a0180fe7d82a8232487 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:01.811366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:01.812718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:01.863156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1002/1964] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:01 2022] POST /identity/v3/users => generated 299 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:01.864190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:01.868800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:01.892619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961fcb4-14c5-472a-b7a8-36531d1ab6ed admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:01.899514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:01.900082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:01.900385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:01.900649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] PATH_INFO: `/v3/users/83aae14cc00a44e7a548dcc2b71ea1ec` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:01.902333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83aae14cc00a44e7a548dcc2b71ea1ec)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:01.904795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:01.950616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74578f43-823d-44ab-bf48-a161d720c5ae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:02.021115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e510590-9e0e-4930-b210-c87f913e912d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:02.022281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510590-9e0e-4930-b210-c87f913e912d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.022661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510590-9e0e-4930-b210-c87f913e912d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.023056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510590-9e0e-4930-b210-c87f913e912d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.025033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e510590-9e0e-4930-b210-c87f913e912d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.085005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e510590-9e0e-4930-b210-c87f913e912d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:02.086885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e510590-9e0e-4930-b210-c87f913e912d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:02.101817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83aae14cc00a44e7a548dcc2b71ea1ec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:02.103449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1003/1965] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:02.115450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.115811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.116112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.143501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:02.143501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.152117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1004/1966] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:02.152907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:02.154202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.170115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c85c1-b76d-4d8c-9b14-01c6141cffed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:02.177066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:02.177847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.180583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.180583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.180583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:02.181032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:02.194033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1005/1967] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:02.194981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:02.197855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.201149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83aae14cc00a44e7a548dcc2b71ea1ec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:02.201958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83aae14cc00a44e7a548dcc2b71ea1ec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:02.219347 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83aae14cc00a44e7a548dcc2b71ea1ec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:02.229192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83aae14cc00a44e7a548dcc2b71ea1ec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:02.241571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 963/1968] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:01 2022] DELETE /identity/v3/users/83aae14cc00a44e7a548dcc2b71ea1ec => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:02.241571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:02.257069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.289347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8da55-cb85-4494-a1e3-d1232a45d3b3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:02.361902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:02.363487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.363934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.364260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.366262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.377284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c1d9e-4447-42a3-af7c-2dd589e949c5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:02.383613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:02.386644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:02.406878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1006/1969] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:02.408109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:02.410045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.410045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.410506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] PATH_INFO: `/v3/projects/1c6b8df0f2664038bf94b7937acc1ff4/users/6e4f2c78a28d4120afa3782994a2613f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.427712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c6b8df0f2664038bf94b7937acc1ff4, user_id=6e4f2c78a28d4120afa3782994a2613f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:02.431268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:02.506628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 964/1970] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:02 2022] PUT /identity/v3/projects/1c6b8df0f2664038bf94b7937acc1ff4/users/6e4f2c78a28d4120afa3782994a2613f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:02.511003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:02.522101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9307720-0d6a-4a78-940e-7a9c10862391 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:02.595346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74d3b0-c93c-4a21-baf0-a70f3542b115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:02.596133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74d3b0-c93c-4a21-baf0-a70f3542b115 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.596423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74d3b0-c93c-4a21-baf0-a70f3542b115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.598738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74d3b0-c93c-4a21-baf0-a70f3542b115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.599531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74d3b0-c93c-4a21-baf0-a70f3542b115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:02.601674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74d3b0-c93c-4a21-baf0-a70f3542b115 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:02.614000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1007/1971] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:02.618363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:02.633218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649243b-c294-4142-bf74-27e54a5f75bc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:02.639057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:02.640223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.640560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.641121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] PATH_INFO: `/v3/projects/1c6b8df0f2664038bf94b7937acc1ff4/users/6e4f2c78a28d4120afa3782994a2613f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.643156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c6b8df0f2664038bf94b7937acc1ff4, user_id=6e4f2c78a28d4120afa3782994a2613f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:02.647606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:02.661566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 965/1972] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:02 2022] PUT /identity/v3/projects/1c6b8df0f2664038bf94b7937acc1ff4/users/6e4f2c78a28d4120afa3782994a2613f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:02.672894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.673214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.674275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.704732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] MFA Rules not processed for user `6e4f2c78a28d4120afa3782994a2613f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:02.707718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.810291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1008/1973] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:02.824024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:02.825908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.886787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:02.888481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:02.961447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331b67-7ea3-4b07-95a1-018121078f2c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:02.985322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:02.987734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:02.988076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:02.990798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:02.991917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:03.043784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcb51b-4918-41ae-a3ac-48ad5ec0bda5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:03.064459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:03.066032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.066414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.066792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] PATH_INFO: `/v3/projects/ef85ed99f5e44d1489e690354654aff3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.068172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef85ed99f5e44d1489e690354654aff3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:03.070124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:03.091893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:03.093374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:03.113881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 966/1974] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:03.114797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:03.116740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:03.131992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f27f0-b920-480c-b092-34eac8d9b39c service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:03.140021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:03.140021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.140021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.140021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] PATH_INFO: `/v3/projects/0b9cf2a3d7894f1f8550e35bf6787ba6` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.141227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9cf2a3d7894f1f8550e35bf6787ba6)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:03.143270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:03.264180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef85ed99f5e44d1489e690354654aff3', 'request_id': 'req-47e91dc3-35bc-43a1-a651-5f73fa645e01'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.282291 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef85ed99f5e44d1489e690354654aff3', 'request_id': 'req-47e91dc3-35bc-43a1-a651-5f73fa645e01'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.290377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9cf2a3d7894f1f8550e35bf6787ba6', 'request_id': 'req-0ff037b9-4362-4406-a73a-68c777d0553a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.300537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef85ed99f5e44d1489e690354654aff3', 'request_id': 'req-47e91dc3-35bc-43a1-a651-5f73fa645e01'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.386908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9cf2a3d7894f1f8550e35bf6787ba6', 'request_id': 'req-0ff037b9-4362-4406-a73a-68c777d0553a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.396060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9cf2a3d7894f1f8550e35bf6787ba6', 'request_id': 'req-0ff037b9-4362-4406-a73a-68c777d0553a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.417119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] The token cache is being invalidate because project 0b9cf2a3d7894f1f8550e35bf6787ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:03.417476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.418873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.423707 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 967/1975] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:03 2022] DELETE /identity/v3/projects/0b9cf2a3d7894f1f8550e35bf6787ba6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:03.424849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:03.426373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:03.502244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff037b9-4362-4406-a73a-68c777d0553a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:03.505836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] The token cache is being invalidate because project ef85ed99f5e44d1489e690354654aff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:03.505836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.505836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47e91dc3-35bc-43a1-a651-5f73fa645e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.506578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1009/1976] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:02 2022] DELETE /identity/v3/projects/ef85ed99f5e44d1489e690354654aff3 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:03.510336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:03.511295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.511869 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.512021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] PATH_INFO: `/v3/users/57312d73a1574184a15f6bd3d56dbfff` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.513504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57312d73a1574184a15f6bd3d56dbfff)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:03.514674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb8aa-88b3-4bbf-bb3a-d6db527043d5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.514972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb8aa-88b3-4bbf-bb3a-d6db527043d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.515253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb8aa-88b3-4bbf-bb3a-d6db527043d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.516607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:03.546361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ee9eb8aa-88b3-4bbf-bb3a-d6db527043d5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:03.547795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ee9eb8aa-88b3-4bbf-bb3a-d6db527043d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:03.556236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1010/1977] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:03.564482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eed51-9fd2-4f80-a80a-eb5af88ea8c7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.566612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eed51-9fd2-4f80-a80a-eb5af88ea8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.567101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eed51-9fd2-4f80-a80a-eb5af88ea8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.591539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d24eed51-9fd2-4f80-a80a-eb5af88ea8c7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:03.593748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d24eed51-9fd2-4f80-a80a-eb5af88ea8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:03.606877 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1011/1978] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:03.611027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24eed51-9fd2-4f80-a80a-eb5af88ea8c7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:03.621412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24eed51-9fd2-4f80-a80a-eb5af88ea8c7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:03.626120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:03.626783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.627062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.627334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.628248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:03.628592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:03.630180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:03.648768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1012/1979] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:16:03.653190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:03.664506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea901e-a24f-45a4-8166-ce31ae211f07 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:03.669819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:03.670558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.670842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.671179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.671905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:03.673995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:03.739950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57312d73a1574184a15f6bd3d56dbfff'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.762851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1013/1980] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:03 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:03.766936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:03.818065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57312d73a1574184a15f6bd3d56dbfff'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.827113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57312d73a1574184a15f6bd3d56dbfff'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.836786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57312d73a1574184a15f6bd3d56dbfff'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.845367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b68d4-4686-41b3-8d19-7589555f77ee admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:03.852443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:03.853629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:03.854103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:03.854575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:03.855278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:03.857240 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:03.866043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57312d73a1574184a15f6bd3d56dbfff'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:03.884608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 968/1981] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:03 2022] DELETE /identity/v3/users/57312d73a1574184a15f6bd3d56dbfff => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:03.955681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1014/1982] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:03 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:03.958604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:03.960077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:04.040061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5bb86-6190-46ad-a20b-07f37e2bcb5c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.061546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.063980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.064428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.064905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.065949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.071615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.087432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 969/1983] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:04.088889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.101242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee7e53-ada0-4370-a54a-698b045ad047 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.115783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3317d7-659d-45f0-8d2e-f9df4f561fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.115783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3317d7-659d-45f0-8d2e-f9df4f561fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.115783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3317d7-659d-45f0-8d2e-f9df4f561fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.115783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3317d7-659d-45f0-8d2e-f9df4f561fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.116814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3317d7-659d-45f0-8d2e-f9df4f561fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.116814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3317d7-659d-45f0-8d2e-f9df4f561fd3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.127816 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1015/1984] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:04.134974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.147416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5d0f1-fab1-4a36-8685-db6585facc7e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.152801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.153692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.154092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.154396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] PATH_INFO: `/v3/projects/7dfe6678fdcb4cd79916dbe8f65c15ec/users/7c62b21595fc46b9a7b084ddb24d1116/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.172774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfe6678fdcb4cd79916dbe8f65c15ec, user_id=7c62b21595fc46b9a7b084ddb24d1116, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.176704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.245229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 970/1985] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:04 2022] PUT /identity/v3/projects/7dfe6678fdcb4cd79916dbe8f65c15ec/users/7c62b21595fc46b9a7b084ddb24d1116/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:04.260034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.260574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.260953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.302520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] MFA Rules not processed for user `7c62b21595fc46b9a7b084ddb24d1116`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:04.303981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:04.379488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1016/1986] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:04.384138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.457498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6459cd-492b-4990-abba-5d19ca308111 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.462450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.463108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.463418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.463739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.464469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.466357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.474947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.476654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:04.491393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e05d5-2bd3-4f2e-b2a8-cb8c41b07713 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.496377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.497385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.497837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.498111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] PATH_INFO: `/v3/projects/2c07684217a64763a036cb4652806dcc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.499492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c07684217a64763a036cb4652806dcc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.501659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.561659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 971/1987] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:04 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:04.566264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.625605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c07684217a64763a036cb4652806dcc', 'request_id': 'req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:04.632556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c07684217a64763a036cb4652806dcc', 'request_id': 'req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:04.635166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db009ec-018d-4c28-b3aa-842eee93dee6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.639137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c07684217a64763a036cb4652806dcc', 'request_id': 'req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:04.692966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.693645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.693928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.694204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.694889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.696894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.708926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] The token cache is being invalidate because project 2c07684217a64763a036cb4652806dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:04.709218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:04.709931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f06ebc1-8456-4ca7-b338-4f0c2220a8f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:04.711512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1017/1988] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:04 2022] DELETE /identity/v3/projects/2c07684217a64763a036cb4652806dcc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:04.716179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbda947-11be-4320-8d45-da3ceab9086a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.716472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbda947-11be-4320-8d45-da3ceab9086a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.716750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbda947-11be-4320-8d45-da3ceab9086a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.735866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7bbda947-11be-4320-8d45-da3ceab9086a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:04.737713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7bbda947-11be-4320-8d45-da3ceab9086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:04.747486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1018/1989] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:04.754955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181945-1bf0-4087-adef-9cd00db31de3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.755323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181945-1bf0-4087-adef-9cd00db31de3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.755751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181945-1bf0-4087-adef-9cd00db31de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.780119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 972/1990] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:04 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:04.780321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c0181945-1bf0-4087-adef-9cd00db31de3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:04.780321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c0181945-1bf0-4087-adef-9cd00db31de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:04.781246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.782995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:04.790009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1019/1991] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:04.796769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e460fa-837e-4a0b-b309-5adb5f7df330 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.797836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181945-1bf0-4087-adef-9cd00db31de3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.803570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.804406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.804766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.805118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.806028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.807744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.809560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181945-1bf0-4087-adef-9cd00db31de3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.818952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.819695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.819992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.820266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.820636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 973/1992] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:04.821099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:04.821422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.823445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.824750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.836059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e69bf-0d7f-43be-978c-a6e1d2d2a349 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.837137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1020/1993] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:16:04.841333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.850249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.851018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.851377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.851796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.852663 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.854370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.855568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a4d8-08a8-4096-a50c-9a6bdda96ac0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.861131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.863891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.863891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.863891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.864879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.867187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.870686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 974/1994] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:04.871747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:04.883489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41a799-1e4a-4310-893d-903a32e60528 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:04.889473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed52f53-944d-43f8-b5e2-f9307b663b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:04.890568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed52f53-944d-43f8-b5e2-f9307b663b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:04.891000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed52f53-944d-43f8-b5e2-f9307b663b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:04.891334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed52f53-944d-43f8-b5e2-f9307b663b8f admin admin] PATH_INFO: `/v3/projects/9958c5ee0f2443968e45c6b26d81844c/users/bb8e6ca05f6b43f4b7c9c07fa8808215/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:04.915467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed52f53-944d-43f8-b5e2-f9307b663b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9958c5ee0f2443968e45c6b26d81844c, user_id=bb8e6ca05f6b43f4b7c9c07fa8808215, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:04.926077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed52f53-944d-43f8-b5e2-f9307b663b8f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:04.982635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1021/1995] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:04 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:04.983999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.052161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6861f1-4955-4e72-8323-1d2e5ed8e0f9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.054454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 975/1996] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:04 2022] PUT /identity/v3/projects/9958c5ee0f2443968e45c6b26d81844c/users/bb8e6ca05f6b43f4b7c9c07fa8808215/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:05.066478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba371f0-7325-4a9f-881a-379d11b76a85 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.066879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba371f0-7325-4a9f-881a-379d11b76a85 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.067397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba371f0-7325-4a9f-881a-379d11b76a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.096373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6ba371f0-7325-4a9f-881a-379d11b76a85 None None] MFA Rules not processed for user `bb8e6ca05f6b43f4b7c9c07fa8808215`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:05.098981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6ba371f0-7325-4a9f-881a-379d11b76a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:05.115376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.116436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.116764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.117258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.120304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.120304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.169390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 976/1997] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:05.173887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba371f0-7325-4a9f-881a-379d11b76a85 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.184485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba371f0-7325-4a9f-881a-379d11b76a85 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.189136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.189899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.190202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.190696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.191275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.193382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.231400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1022/1998] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:04 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:05.235478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.246145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8e514-7d34-45b5-87d2-be3e162c65e8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.262296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.262985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.263971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.264336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.265164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.267113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.279590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1023/1999] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:05.285637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.297478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c20a84-533d-4079-bfce-16f81de3f41f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.302962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.303806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.304104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.304399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.305160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.306992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.307849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 977/2000] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:05 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:05.313518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.333700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1024/2001] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:05.334386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.336154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:05.387836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03235-05d6-4702-8d14-266449906292 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.396093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.397028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.397314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.397592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.398293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.400280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.402329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e473f-1fe7-4ad3-a3cd-4c13d46b055a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.465788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.466446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.466727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.467008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.468821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:05.524706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 978/2002] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:05 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:05.525808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.533919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.536468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef91d0-33a1-48e5-9ff7-0e541ecf559c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.536583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.542000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0977ae-4bbc-44e7-a4c1-f2c142035afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.542867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0977ae-4bbc-44e7-a4c1-f2c142035afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.543152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0977ae-4bbc-44e7-a4c1-f2c142035afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.544205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0977ae-4bbc-44e7-a4c1-f2c142035afe admin admin] PATH_INFO: `/v3/projects/6cd508f803f94c8595567745915a86f9/users/e93e8247840d4e6ea96c99323c4d639e/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.548371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1025/2003] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:05.549283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.560166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0977ae-4bbc-44e7-a4c1-f2c142035afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd508f803f94c8595567745915a86f9, user_id=e93e8247840d4e6ea96c99323c4d639e, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.560789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b4b5-1e08-4021-88c0-4a8afae779ac service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.563539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0977ae-4bbc-44e7-a4c1-f2c142035afe admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.574651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.574651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.574651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.574651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.575035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.575035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.594748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1026/2004] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:05.596739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.609734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b84f65-6e1c-4a77-8164-a5f154ad2abc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.616384 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.617505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.617969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.618420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] PATH_INFO: `/v3/projects/dd74fbe35c314756a042b06bdae9cfc9/users/d6a231893f9f4b83baea2aecdd1458b4/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.638323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd74fbe35c314756a042b06bdae9cfc9, user_id=d6a231893f9f4b83baea2aecdd1458b4, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.641270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.677174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 979/2005] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:05 2022] PUT /identity/v3/projects/6cd508f803f94c8595567745915a86f9/users/e93e8247840d4e6ea96c99323c4d639e/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:05.687304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bfadd-11c2-4bd0-b5cd-4c0f3ebf4d67 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.687596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bfadd-11c2-4bd0-b5cd-4c0f3ebf4d67 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.687940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bfadd-11c2-4bd0-b5cd-4c0f3ebf4d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.715649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-561bfadd-11c2-4bd0-b5cd-4c0f3ebf4d67 None None] MFA Rules not processed for user `e93e8247840d4e6ea96c99323c4d639e`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:05.716872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-561bfadd-11c2-4bd0-b5cd-4c0f3ebf4d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:05.752415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1027/2006] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:05 2022] PUT /identity/v3/projects/dd74fbe35c314756a042b06bdae9cfc9/users/d6a231893f9f4b83baea2aecdd1458b4/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:05.756719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.808093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 980/2007] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:05.812969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bfadd-11c2-4bd0-b5cd-4c0f3ebf4d67 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.875097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf5562-adad-46cc-9672-8b79acde8f45 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.881106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.881994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.882281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.882558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.883289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.884953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.900974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1028/2008] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:05.902449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:05.918926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bfadd-11c2-4bd0-b5cd-4c0f3ebf4d67 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.925739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607f801-9490-4ba2-b572-c41ab71295c1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:05.929362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825664b1-d6e6-425f-b1f4-5df14ad5881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.930242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825664b1-d6e6-425f-b1f4-5df14ad5881e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.930495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:05.930608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825664b1-d6e6-425f-b1f4-5df14ad5881e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.930794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825664b1-d6e6-425f-b1f4-5df14ad5881e admin admin] PATH_INFO: `/v3/projects/dd74fbe35c314756a042b06bdae9cfc9/users/d6a231893f9f4b83baea2aecdd1458b4/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.931218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.931508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.931903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.932712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.932890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825664b1-d6e6-425f-b1f4-5df14ad5881e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd74fbe35c314756a042b06bdae9cfc9, user_id=d6a231893f9f4b83baea2aecdd1458b4, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:05.934724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.936196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825664b1-d6e6-425f-b1f4-5df14ad5881e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:05.947756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1029/2009] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:05 2022] PUT /identity/v3/projects/dd74fbe35c314756a042b06bdae9cfc9/users/d6a231893f9f4b83baea2aecdd1458b4/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:05.958992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:05.959433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:05.959819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:05.986813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] MFA Rules not processed for user `d6a231893f9f4b83baea2aecdd1458b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:05.988890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:06.066451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 981/2010] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:05 2022] POST /identity/v3/projects => generated 381 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:06.070564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.073733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1030/2011] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:06.119948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.121706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:06.172256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08407a-b49a-40b0-b633-44f779970c74 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.177700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.178389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.178674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.178946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.179674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.181626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.203265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e574193-d801-4590-a94a-4c4e0084b59d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.263721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.264835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.265400 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.265659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.283124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 982/2012] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:06 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:06.284302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.303273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f6ff0-be95-4348-9d60-e7e2c5bdac92 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.310698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.312238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.312238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.312736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.314431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.316834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.341598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 983/2013] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:06.343225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.344431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.344867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.357870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1031/2014] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:06.364476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2db63c-3a57-4187-acbf-9f1f7edabdaa admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.370129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.371396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.371987 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.372397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] PATH_INFO: `/v3/projects/81d056c1bad2484fb057ddccffb14872/users/382496c899d5494ab1e8f15def267944/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.388916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d056c1bad2484fb057ddccffb14872, user_id=382496c899d5494ab1e8f15def267944, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.391901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.479185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 984/2015] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:06 2022] PUT /identity/v3/projects/81d056c1bad2484fb057ddccffb14872/users/382496c899d5494ab1e8f15def267944/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:06.480582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.494643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b765c0c-cbe0-410a-b2c3-071a320e3ac7 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.580392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.596025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.597768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.597768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.599124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.599124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.600906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.620311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1032/2016] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:06.624261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.634714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5b1c0-b851-4940-a0e7-39a034475ffb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.639503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a8b3b-9a3e-4979-82fd-5a45e7fcdb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.640443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8b3b-9a3e-4979-82fd-5a45e7fcdb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.640724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8b3b-9a3e-4979-82fd-5a45e7fcdb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.641133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8b3b-9a3e-4979-82fd-5a45e7fcdb27 admin admin] PATH_INFO: `/v3/projects/81d056c1bad2484fb057ddccffb14872/users/382496c899d5494ab1e8f15def267944/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.643223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a8b3b-9a3e-4979-82fd-5a45e7fcdb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d056c1bad2484fb057ddccffb14872, user_id=382496c899d5494ab1e8f15def267944, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.645406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be0e8e-a267-4bb3-b8a3-c0ee589da9d4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.645958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a8b3b-9a3e-4979-82fd-5a45e7fcdb27 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.649267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.649927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.650259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.650551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.652603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:06.657907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1033/2017] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:06 2022] PUT /identity/v3/projects/81d056c1bad2484fb057ddccffb14872/users/382496c899d5494ab1e8f15def267944/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:06.669061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7fc4e-b44c-4857-81e4-5ab1338b816f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.669350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7fc4e-b44c-4857-81e4-5ab1338b816f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.669631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7fc4e-b44c-4857-81e4-5ab1338b816f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.704132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-17a7fc4e-b44c-4857-81e4-5ab1338b816f None None] MFA Rules not processed for user `382496c899d5494ab1e8f15def267944`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:06.705438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-17a7fc4e-b44c-4857-81e4-5ab1338b816f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:06.726434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.728349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.738720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 985/2018] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:16:06.796411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1034/2019] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:06.883064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:06.897349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3c7a9-7041-4694-bca4-38424f342276 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:06.948601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:06.949365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:06.949648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:06.949921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:06.959900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:06.961459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:06.971301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 986/2020] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:07.318800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7fc4e-b44c-4857-81e4-5ab1338b816f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:07.328054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7fc4e-b44c-4857-81e4-5ab1338b816f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:07.332288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:07.332924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:07.333204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:07.333478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:07.395031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:07.396543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:07.406608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1035/2021] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:07.616665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:07.632215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbc1a-4617-4c90-885b-585ba25cd87a service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:07.635846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:07.636490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:07.636772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:07.637049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:07.638835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:07.719074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:07.720883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:07.734244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 987/2022] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:08.145551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:08.155859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20ff0a-6f83-4884-84c6-02bfc77d4a54 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:08.160841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee8a3e-776a-48f6-961a-6be244728dd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:08.161811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8a3e-776a-48f6-961a-6be244728dd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:08.162248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8a3e-776a-48f6-961a-6be244728dd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:08.162552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8a3e-776a-48f6-961a-6be244728dd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:08.164871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f9ee8a3e-776a-48f6-961a-6be244728dd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:08.229097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee8a3e-776a-48f6-961a-6be244728dd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:08.230783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee8a3e-776a-48f6-961a-6be244728dd6 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:08.242020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1036/2023] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:16:08.276968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:08.358717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09ac8e-4400-428f-a01e-f1ac9e57a7dc service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:08.365258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:08.366283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:08.366543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:08.366804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] PATH_INFO: `/v3/users/e93e8247840d4e6ea96c99323c4d639e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:08.368035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93e8247840d4e6ea96c99323c4d639e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:08.370005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:08.572178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93e8247840d4e6ea96c99323c4d639e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:08.634838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93e8247840d4e6ea96c99323c4d639e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:08.636467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93e8247840d4e6ea96c99323c4d639e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:08.651893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93e8247840d4e6ea96c99323c4d639e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:08.660674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93e8247840d4e6ea96c99323c4d639e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:08.669921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 988/2024] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:08 2022] DELETE /identity/v3/users/e93e8247840d4e6ea96c99323c4d639e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:08.674347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:08.675485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:08.676039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:08.700628 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:08.701083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:08.813505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1037/2025] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:08.825584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:08.827405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:08.931806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a95ab-d09c-4453-8808-0ff3d4218f8b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:08.935417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:08.936148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:08.936433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:08.936708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:08.946284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:08.957439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:08.960513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 989/2026] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:09.483838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:09.485297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:09.510762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735bb84-1448-48d0-b048-e8e074efc45f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:09.519689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:09.520547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:09.520834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:09.521104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] PATH_INFO: `/v3/projects/6cd508f803f94c8595567745915a86f9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:09.522555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd508f803f94c8595567745915a86f9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:09.524534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:09.661657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd508f803f94c8595567745915a86f9', 'request_id': 'req-1b468017-8d59-482f-a63e-793044232c0f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:09.670959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd508f803f94c8595567745915a86f9', 'request_id': 'req-1b468017-8d59-482f-a63e-793044232c0f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:09.679319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd508f803f94c8595567745915a86f9', 'request_id': 'req-1b468017-8d59-482f-a63e-793044232c0f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:09.746113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] The token cache is being invalidate because project 6cd508f803f94c8595567745915a86f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:09.746113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:09.746863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:09.749305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1038/2027] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:09 2022] DELETE /identity/v3/projects/6cd508f803f94c8595567745915a86f9 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:09.754516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:09.764053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d2d66-6802-461e-a764-c8a5d3ac4f85 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:09.820494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:09.821358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:09.821638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:09.821909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] PATH_INFO: `/v3/users/382496c899d5494ab1e8f15def267944` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:09.823115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382496c899d5494ab1e8f15def267944)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:09.825084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:10.002395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382496c899d5494ab1e8f15def267944'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.143117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382496c899d5494ab1e8f15def267944'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.144018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382496c899d5494ab1e8f15def267944'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.157720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382496c899d5494ab1e8f15def267944'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.164990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6cbf169a-0ea7-48d6-897a-90fa50f90471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382496c899d5494ab1e8f15def267944'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.170941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 990/2028] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:09 2022] DELETE /identity/v3/users/382496c899d5494ab1e8f15def267944 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:10.616440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:10.617882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:10.632637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468017-8d59-482f-a63e-793044232c0f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:10.708176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:10.709047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:10.709325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:10.709594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] PATH_INFO: `/v3/projects/81d056c1bad2484fb057ddccffb14872` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:10.710792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d056c1bad2484fb057ddccffb14872)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:10.712795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:10.837742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d056c1bad2484fb057ddccffb14872', 'request_id': 'req-50639d49-714b-43a3-98ce-23765a2a216a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.843460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d056c1bad2484fb057ddccffb14872', 'request_id': 'req-50639d49-714b-43a3-98ce-23765a2a216a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.851446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d056c1bad2484fb057ddccffb14872', 'request_id': 'req-50639d49-714b-43a3-98ce-23765a2a216a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.950655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] The token cache is being invalidate because project 81d056c1bad2484fb057ddccffb14872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:10.950994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.951730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-50639d49-714b-43a3-98ce-23765a2a216a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:10.953349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1039/2029] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:10 2022] DELETE /identity/v3/projects/81d056c1bad2484fb057ddccffb14872 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:10.958561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d493e-b975-4513-bcc0-48363d1f8199 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:10.959085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d493e-b975-4513-bcc0-48363d1f8199 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:10.959176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d493e-b975-4513-bcc0-48363d1f8199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:10.978939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-646d493e-b975-4513-bcc0-48363d1f8199 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:10.980265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-646d493e-b975-4513-bcc0-48363d1f8199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:11.040075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 991/2030] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:11.047651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59563f18-97f1-415d-bfc5-13c8b4992fa0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.047965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59563f18-97f1-415d-bfc5-13c8b4992fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.048258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59563f18-97f1-415d-bfc5-13c8b4992fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.067871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-59563f18-97f1-415d-bfc5-13c8b4992fa0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:11.069082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-59563f18-97f1-415d-bfc5-13c8b4992fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:11.077100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1040/2031] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:11.081782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d493e-b975-4513-bcc0-48363d1f8199 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.091268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d493e-b975-4513-bcc0-48363d1f8199 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.096289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.097490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.097766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.098035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.098856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:11.099157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.100824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.113081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 992/2032] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:11.117536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59563f18-97f1-415d-bfc5-13c8b4992fa0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.126609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59563f18-97f1-415d-bfc5-13c8b4992fa0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.131591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.132273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.132559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.132836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.133532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.135443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.235555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1041/2033] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:11 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:11.240036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.249794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52b24a-5193-41e2-af51-07984396e8bb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.307776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.311844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.311844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.311844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.311844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.311844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.394787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 993/2034] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:11 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:11.401452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.416677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f66f-8d5e-4702-b025-6fe0d7100c35 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.421672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.422375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.422660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.422934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.423692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.426083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.436822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1042/2035] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:11.441737 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.451997 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868a6e9-e752-49bd-9c46-5ee8386ea387 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.457960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef3358-cec0-418c-96fb-b38f0f5db59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.458635 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3358-cec0-418c-96fb-b38f0f5db59c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.459466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3358-cec0-418c-96fb-b38f0f5db59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.459466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3358-cec0-418c-96fb-b38f0f5db59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.460207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef3358-cec0-418c-96fb-b38f0f5db59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.461610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef3358-cec0-418c-96fb-b38f0f5db59c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.472488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 994/2036] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:11.476723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.486681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963b303-daac-486a-9f0d-f843784fb69f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.492752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.493727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.494082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.494429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] PATH_INFO: `/v3/projects/cc1e9a564da046168038188d4f932720/users/2272548a065f45cf877e51be78d0d471/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.510084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1e9a564da046168038188d4f932720, user_id=2272548a065f45cf877e51be78d0d471, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.512892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.591845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1043/2037] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:11 2022] PUT /identity/v3/projects/cc1e9a564da046168038188d4f932720/users/2272548a065f45cf877e51be78d0d471/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:11.592768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.593061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.593336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.623188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] MFA Rules not processed for user `2272548a065f45cf877e51be78d0d471`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:11.624456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:11.686844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 995/2038] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:11.691519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.754494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc61f6-bfce-4254-b921-0535719cc4ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.755719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.757283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:11.759503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.760352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.760656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.760941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.761737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.763883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.773925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ed4d0-53df-4a69-abf6-04ae2f9c7135 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.778971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.779911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.780235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.780573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] PATH_INFO: `/v3/users/7c62b21595fc46b9a7b084ddb24d1116` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.781898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c62b21595fc46b9a7b084ddb24d1116)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.784515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.844444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1044/2039] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:11 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:11.848994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:11.911100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3387220-0075-4acd-8a91-9cdacebadfe6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:11.915864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:11.916560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:11.916843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:11.917118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:11.917798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:11.919756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:11.956476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c62b21595fc46b9a7b084ddb24d1116'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:12.069244 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1045/2040] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:11 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:12.070845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.081883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab21ba-b714-4f9d-8208-c86e54883537 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.087603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.088606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.088889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.089164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.089280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c62b21595fc46b9a7b084ddb24d1116'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:12.089904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.090149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c62b21595fc46b9a7b084ddb24d1116'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:12.091504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.105567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1046/2041] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:12.109392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.112361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.115525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c62b21595fc46b9a7b084ddb24d1116'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:12.122952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-21190530-68a4-464a-ac87-92b9235451fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c62b21595fc46b9a7b084ddb24d1116'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:12.129398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7a6aa-0511-439b-a176-075996104bc6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.132172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 996/2042] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:11 2022] DELETE /identity/v3/users/7c62b21595fc46b9a7b084ddb24d1116 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:12.136403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.137165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.137645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.137870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.138813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.139106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.139382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] PATH_INFO: `/v3/projects/2dcf5e7a61384badb2f10b4d053cd9e2/users/322cb0d6ded44c2aaa81a695e75da922/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.157225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcf5e7a61384badb2f10b4d053cd9e2, user_id=322cb0d6ded44c2aaa81a695e75da922, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.161753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.164542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:12.165794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.220800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 997/2043] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:12.231331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.232850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.277697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1047/2044] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:12 2022] PUT /identity/v3/projects/2dcf5e7a61384badb2f10b4d053cd9e2/users/322cb0d6ded44c2aaa81a695e75da922/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:12.281778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.288940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5308-4df5-45c2-8d98-c709c213dfb7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.339976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.340706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.340993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.341267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.346935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfa916-97a7-444a-9001-9a8217db0407 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.351670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.356793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549c2ed-0586-45ae-992e-125853989817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.356793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549c2ed-0586-45ae-992e-125853989817 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.356995 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.357080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549c2ed-0586-45ae-992e-125853989817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.357080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549c2ed-0586-45ae-992e-125853989817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.357080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549c2ed-0586-45ae-992e-125853989817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.357080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549c2ed-0586-45ae-992e-125853989817 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.364013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 998/2045] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:16:12.367318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1048/2046] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:12.371384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.381653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4aed6-ea52-42a2-b050-c4ef2e337528 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.386306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.387185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.387493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.387881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] PATH_INFO: `/v3/projects/2dcf5e7a61384badb2f10b4d053cd9e2/users/322cb0d6ded44c2aaa81a695e75da922/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.389822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcf5e7a61384badb2f10b4d053cd9e2, user_id=322cb0d6ded44c2aaa81a695e75da922, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.392579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.403257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 999/2047] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:12 2022] PUT /identity/v3/projects/2dcf5e7a61384badb2f10b4d053cd9e2/users/322cb0d6ded44c2aaa81a695e75da922/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:12.415236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.415529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.415837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.447824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] MFA Rules not processed for user `322cb0d6ded44c2aaa81a695e75da922`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:12.449024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.515268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1049/2048] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:12.526940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.528644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.602881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94525-acfa-4325-a587-277e473c2725 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.607219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.608027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.608317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.608672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.610665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.652573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.654675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.697043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.699115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.710121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1000/2049] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:12.739964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65423ae1-c0aa-40da-9cac-aed2cba8ee79 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.744715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.745599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.746018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.746301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] PATH_INFO: `/v3/users/2b3b3ae884074afb9ad8f62467b5de06` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.747554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3b3ae884074afb9ad8f62467b5de06)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.749902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.844245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:12.845620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:12.876937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c500c-0906-49a0-9f08-b00fe168f302 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:12.882119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:12.882994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:12.883280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:12.883552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] PATH_INFO: `/v3/projects/7dfe6678fdcb4cd79916dbe8f65c15ec` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:12.884794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfe6678fdcb4cd79916dbe8f65c15ec)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:12.886996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:12.951323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3b3ae884074afb9ad8f62467b5de06'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.023032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfe6678fdcb4cd79916dbe8f65c15ec', 'request_id': 'req-cd20d5ab-98f3-43e4-a59e-68231448baee'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.053306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3b3ae884074afb9ad8f62467b5de06'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.061362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3b3ae884074afb9ad8f62467b5de06'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.062130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3b3ae884074afb9ad8f62467b5de06'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.074559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1b0ca2bb-549d-42ff-8631-9fb93aab375f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3b3ae884074afb9ad8f62467b5de06'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.081270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1050/2050] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:12 2022] DELETE /identity/v3/users/2b3b3ae884074afb9ad8f62467b5de06 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:13.085402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:13.085665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:13.085940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:13.105132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:13.105694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:13.160570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfe6678fdcb4cd79916dbe8f65c15ec', 'request_id': 'req-cd20d5ab-98f3-43e4-a59e-68231448baee'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.164298 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfe6678fdcb4cd79916dbe8f65c15ec', 'request_id': 'req-cd20d5ab-98f3-43e4-a59e-68231448baee'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.177932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] The token cache is being invalidate because project 7dfe6678fdcb4cd79916dbe8f65c15ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:13.178459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.179191 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.183590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1051/2052] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:13.187745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:13.189116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1001/2052] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:12 2022] DELETE /identity/v3/projects/7dfe6678fdcb4cd79916dbe8f65c15ec => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:13.190534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:13.192685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:13.195548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:13.210010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dbc91-752f-4e31-b4de-a0c5e2e75575 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:13.217266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:13.217266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:13.217266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:13.217266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] PATH_INFO: `/v3/users/11f5dba4ab4140b18d06c1e6bd86d5ea` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:13.219247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f5dba4ab4140b18d06c1e6bd86d5ea)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:13.220752 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:13.267235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20d5ab-98f3-43e4-a59e-68231448baee admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:13.272886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:13.273823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:13.274108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:13.274379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] PATH_INFO: `/v3/users/bb8e6ca05f6b43f4b7c9c07fa8808215` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:13.275579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8e6ca05f6b43f4b7c9c07fa8808215)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:13.277585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:13.446760 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f5dba4ab4140b18d06c1e6bd86d5ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.471453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8e6ca05f6b43f4b7c9c07fa8808215'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.545655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f5dba4ab4140b18d06c1e6bd86d5ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.553821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f5dba4ab4140b18d06c1e6bd86d5ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.554782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f5dba4ab4140b18d06c1e6bd86d5ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.569404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f5dba4ab4140b18d06c1e6bd86d5ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.575623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1052/2053] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:13 2022] DELETE /identity/v3/users/11f5dba4ab4140b18d06c1e6bd86d5ea => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:13.576419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:13.577729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:13.630905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8e6ca05f6b43f4b7c9c07fa8808215'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.635440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8e6ca05f6b43f4b7c9c07fa8808215'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.639914 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a59c59-0efb-4fef-b2b6-04a1cbf26e4b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:13.645261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:13.647247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:13.647247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:13.647247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:13.648794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:13.658362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8e6ca05f6b43f4b7c9c07fa8808215'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.666340 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ee1dc05c-4d1d-4c2f-b42f-29408a842aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8e6ca05f6b43f4b7c9c07fa8808215'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:13.674377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1002/2054] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:13 2022] DELETE /identity/v3/users/bb8e6ca05f6b43f4b7c9c07fa8808215 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:13.693203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:13.693408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:13.693691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:13.731247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:13.733805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:13.754817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:13.759664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:13.784418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1053/2055] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:13.811381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1003/2056] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:13.833258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:13.834810 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:13.847433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ec88-18c9-46fd-950b-6e35be66aaae service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:13.903755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:13.904415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:13.904700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:13.904974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:13.916214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:13.917727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:13.928198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1054/2057] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:14.250341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:14.252076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:14.267214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73838813-6c35-45db-81f5-a6d57aeab798 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:14.275808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:14.275808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:14.276014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:14.276498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] PATH_INFO: `/v3/projects/9958c5ee0f2443968e45c6b26d81844c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:14.277998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] RBAC: Authorizing `identity:delete_project(project_id=9958c5ee0f2443968e45c6b26d81844c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:14.279993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:14.333722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:14.335178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:14.348082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7247649-329e-4d21-bddb-8620e08f3fdc service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:14.353473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:14.354251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:14.354532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:14.361447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] PATH_INFO: `/v3/projects/3f6a70620e7b452b8fdecd66348694d2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:14.361447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6a70620e7b452b8fdecd66348694d2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:14.361447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:14.401288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9958c5ee0f2443968e45c6b26d81844c', 'request_id': 'req-49fbe15f-8804-4241-a403-1ffe3251cead'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.548702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9958c5ee0f2443968e45c6b26d81844c', 'request_id': 'req-49fbe15f-8804-4241-a403-1ffe3251cead'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.555437 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9958c5ee0f2443968e45c6b26d81844c', 'request_id': 'req-49fbe15f-8804-4241-a403-1ffe3251cead'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.570451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] The token cache is being invalidate because project 9958c5ee0f2443968e45c6b26d81844c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:14.570749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.571417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.573054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1004/2058] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:14 2022] DELETE /identity/v3/projects/9958c5ee0f2443968e45c6b26d81844c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:14.576093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:14.577647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:14.586537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6a70620e7b452b8fdecd66348694d2', 'request_id': 'req-31ce1946-6baa-4452-af16-b8462ffd7ac4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.594998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6a70620e7b452b8fdecd66348694d2', 'request_id': 'req-31ce1946-6baa-4452-af16-b8462ffd7ac4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.602556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6a70620e7b452b8fdecd66348694d2', 'request_id': 'req-31ce1946-6baa-4452-af16-b8462ffd7ac4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.651512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe15f-8804-4241-a403-1ffe3251cead admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:14.709674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:14.710534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:14.710808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:14.711076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] PATH_INFO: `/v3/projects/512f2c902c654000bf31972fcd402137` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:14.712275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] RBAC: Authorizing `identity:delete_project(project_id=512f2c902c654000bf31972fcd402137)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:14.714168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:14.910940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] The token cache is being invalidate because project 3f6a70620e7b452b8fdecd66348694d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:14.916023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.916023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:14.916023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1055/2059] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:14 2022] DELETE /identity/v3/projects/3f6a70620e7b452b8fdecd66348694d2 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:14.916023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:14.916023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:14.931080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce1946-6baa-4452-af16-b8462ffd7ac4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:14.934529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:14.934937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:14.935221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:14.935495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] PATH_INFO: `/v3/users/d6a231893f9f4b83baea2aecdd1458b4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:14.936876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a231893f9f4b83baea2aecdd1458b4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:14.990568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:14.991941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512f2c902c654000bf31972fcd402137', 'request_id': 'req-0ead592b-8581-40e1-82c7-5715aac6838e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.174137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512f2c902c654000bf31972fcd402137', 'request_id': 'req-0ead592b-8581-40e1-82c7-5715aac6838e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.180290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512f2c902c654000bf31972fcd402137', 'request_id': 'req-0ead592b-8581-40e1-82c7-5715aac6838e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.191273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] The token cache is being invalidate because project 512f2c902c654000bf31972fcd402137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:15.191579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.192317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.193900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1005/2060] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:14 2022] DELETE /identity/v3/projects/512f2c902c654000bf31972fcd402137 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:15.194668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:15.195967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:15.257664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead592b-8581-40e1-82c7-5715aac6838e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:15.262638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:15.263779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:15.264122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:15.264406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] PATH_INFO: `/v3/users/ca93062b9ba843adb714ecfa1fb8b7cc` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:15.265634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca93062b9ba843adb714ecfa1fb8b7cc)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:15.268678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:15.335199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a231893f9f4b83baea2aecdd1458b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.399796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a231893f9f4b83baea2aecdd1458b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.407900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a231893f9f4b83baea2aecdd1458b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.409250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a231893f9f4b83baea2aecdd1458b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.422464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a231893f9f4b83baea2aecdd1458b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.429967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1056/2061] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:14 2022] DELETE /identity/v3/users/d6a231893f9f4b83baea2aecdd1458b4 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:15.430852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:15.432290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:15.474991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca93062b9ba843adb714ecfa1fb8b7cc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.510763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3206c2-515c-4710-aed8-04957258dee0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:15.515683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:15.516569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:15.516845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:15.517116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] PATH_INFO: `/v3/users/cd49030cbb5943e1b31b27c10f753d5d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:15.519080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd49030cbb5943e1b31b27c10f753d5d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:15.521156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:15.539782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca93062b9ba843adb714ecfa1fb8b7cc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.540444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca93062b9ba843adb714ecfa1fb8b7cc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.553331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca93062b9ba843adb714ecfa1fb8b7cc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.559950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca93062b9ba843adb714ecfa1fb8b7cc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.567258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1006/2062] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:15 2022] DELETE /identity/v3/users/ca93062b9ba843adb714ecfa1fb8b7cc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:15.742761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd49030cbb5943e1b31b27c10f753d5d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.814333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd49030cbb5943e1b31b27c10f753d5d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.823211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd49030cbb5943e1b31b27c10f753d5d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.823864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd49030cbb5943e1b31b27c10f753d5d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.852164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd49030cbb5943e1b31b27c10f753d5d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:15.863772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1057/2063] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:15 2022] DELETE /identity/v3/users/cd49030cbb5943e1b31b27c10f753d5d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:16.041894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:16.043360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:16.108811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1effc-bcc9-4b13-8767-179686564f4f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:16.142269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:16.144735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:16.198593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:16.199538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:16.199863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:16.200342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] PATH_INFO: `/v3/projects/dd74fbe35c314756a042b06bdae9cfc9` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:16.201530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd74fbe35c314756a042b06bdae9cfc9)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:16.203678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:16.262367 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c318e2-b678-4097-b803-f30306cad3d4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:16.267603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:16.269399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:16.270213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:16.270574 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] PATH_INFO: `/v3/projects/4b9e8702bb9a4f108d6aeeca485b8b69` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:16.272164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9e8702bb9a4f108d6aeeca485b8b69)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:16.274316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:16.366002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd74fbe35c314756a042b06bdae9cfc9', 'request_id': 'req-1ed8653e-78a2-46a9-9f55-8a08204a551a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.399470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9e8702bb9a4f108d6aeeca485b8b69', 'request_id': 'req-ede23afc-22d6-49ad-a074-f6fc702bbe79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.404768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9e8702bb9a4f108d6aeeca485b8b69', 'request_id': 'req-ede23afc-22d6-49ad-a074-f6fc702bbe79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.411345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9e8702bb9a4f108d6aeeca485b8b69', 'request_id': 'req-ede23afc-22d6-49ad-a074-f6fc702bbe79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.508219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd74fbe35c314756a042b06bdae9cfc9', 'request_id': 'req-1ed8653e-78a2-46a9-9f55-8a08204a551a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.514123 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd74fbe35c314756a042b06bdae9cfc9', 'request_id': 'req-1ed8653e-78a2-46a9-9f55-8a08204a551a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.525836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] The token cache is being invalidate because project dd74fbe35c314756a042b06bdae9cfc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:16.526115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.526739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.528307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1007/2064] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:16 2022] DELETE /identity/v3/projects/dd74fbe35c314756a042b06bdae9cfc9 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:16.529324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:16.530586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:16.576092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] The token cache is being invalidate because project 4b9e8702bb9a4f108d6aeeca485b8b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:16.576378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.576997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ede23afc-22d6-49ad-a074-f6fc702bbe79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.578541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1058/2065] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:16 2022] DELETE /identity/v3/projects/4b9e8702bb9a4f108d6aeeca485b8b69 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:16.579824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:16.580108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:16.580381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:16.594932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8653e-78a2-46a9-9f55-8a08204a551a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:16.599404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:16.608798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:16.653119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:16.654078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:16.654361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:16.654636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] PATH_INFO: `/v3/projects/78052f5b06774f39abc51a5d70324ae4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:16.655990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78052f5b06774f39abc51a5d70324ae4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:16.657903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:16.660300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1059/2066] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:16.661301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:16.662624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:16.674564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e1b5f-ad4f-408a-9a0a-4d42dccb2a57 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:16.679938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:16.680837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:16.681119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:16.681394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] PATH_INFO: `/v3/users/d8ef9790b34f406f83d6d782920e7544` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:16.682697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8ef9790b34f406f83d6d782920e7544)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:16.684681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:16.904507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78052f5b06774f39abc51a5d70324ae4', 'request_id': 'req-77b27c07-f8e7-49a9-8108-642351771864'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.904877 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8ef9790b34f406f83d6d782920e7544'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.968827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78052f5b06774f39abc51a5d70324ae4', 'request_id': 'req-77b27c07-f8e7-49a9-8108-642351771864'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.974861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78052f5b06774f39abc51a5d70324ae4', 'request_id': 'req-77b27c07-f8e7-49a9-8108-642351771864'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.985885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] The token cache is being invalidate because project 78052f5b06774f39abc51a5d70324ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:16.986169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.986805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-77b27c07-f8e7-49a9-8108-642351771864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:16.989018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1008/2067] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:16 2022] DELETE /identity/v3/projects/78052f5b06774f39abc51a5d70324ae4 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:16.990519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:16.990808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:16.991088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.009250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:17.010462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:17.070138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1009/2068] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:17.071070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.074509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:17.087075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35bebe-5379-49cf-99c4-24c7afddc32d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.093064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.093064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.093064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.093296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] PATH_INFO: `/v3/users/6e4f2c78a28d4120afa3782994a2613f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.094376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4f2c78a28d4120afa3782994a2613f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.096361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.131731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8ef9790b34f406f83d6d782920e7544'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.139390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8ef9790b34f406f83d6d782920e7544'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.140138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8ef9790b34f406f83d6d782920e7544'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.153141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8ef9790b34f406f83d6d782920e7544'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.160205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1060/2069] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:16 2022] DELETE /identity/v3/users/d8ef9790b34f406f83d6d782920e7544 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:17.163464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.170739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:17.239121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6724dcb-eede-4239-96b0-d33bda0908dc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.244114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.244819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.245197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.249406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.249406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:17.249406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.249406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.260621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1061/2070] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:17.264889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.275043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1111d-561f-4c41-b4db-cca3819eacfa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.279724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.280372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.280654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.280926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.281603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.283515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.312708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4f2c78a28d4120afa3782994a2613f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.406417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4f2c78a28d4120afa3782994a2613f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.408016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4f2c78a28d4120afa3782994a2613f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.418351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1062/2071] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:17 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:17.422673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.426152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:17.434694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4f2c78a28d4120afa3782994a2613f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.448506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4f2c78a28d4120afa3782994a2613f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.455819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1010/2072] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:17 2022] DELETE /identity/v3/users/6e4f2c78a28d4120afa3782994a2613f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:17.507650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f80e2-ff1a-4f46-a80d-8ed12dd5e5e1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.582196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.582975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.583316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.583619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.584680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.586760 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.670372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1063/2073] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:17 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:17.675793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.687105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e7a7-e7f1-4c27-bf2c-392144b3a769 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.694961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.695664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.695935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.696215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.697027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.698550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.710391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1011/2074] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:17.712989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.723708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc892-2587-482e-8fab-43c092ddb397 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.728749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.729499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.729853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.735409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.735409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.735409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.743477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1064/2075] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:17.747564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.758032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b5796-1f05-42b9-a464-22d9bb73e7d2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.762630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c75fa-324d-4b60-9825-194b88427108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.763615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c75fa-324d-4b60-9825-194b88427108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.764174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c75fa-324d-4b60-9825-194b88427108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.764277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c75fa-324d-4b60-9825-194b88427108 admin admin] PATH_INFO: `/v3/projects/a5799c76cf744fb2a1f3d73cf6f70c3e/users/ee67269a94d6494597cdf27b19c2f371/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.779920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c75fa-324d-4b60-9825-194b88427108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5799c76cf744fb2a1f3d73cf6f70c3e, user_id=ee67269a94d6494597cdf27b19c2f371, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.782742 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c75fa-324d-4b60-9825-194b88427108 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.802860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:17.805294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:17.818527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8bd23-ef14-4e1d-9961-721f7916d3cf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:17.825891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:17.826920 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.827366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.827876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] PATH_INFO: `/v3/projects/9848d27b0c3f4d02b827cce6050866ca` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.829676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9848d27b0c3f4d02b827cce6050866ca)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:17.832369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:17.869640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1012/2076] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:17 2022] PUT /identity/v3/projects/a5799c76cf744fb2a1f3d73cf6f70c3e/users/ee67269a94d6494597cdf27b19c2f371/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:17.881871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:17.882241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:17.882588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:17.909461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] MFA Rules not processed for user `ee67269a94d6494597cdf27b19c2f371`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:17.910927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:17.987996 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9848d27b0c3f4d02b827cce6050866ca', 'request_id': 'req-1fdbcaf2-861e-4359-87e5-1d397ebc742a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.995225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9848d27b0c3f4d02b827cce6050866ca', 'request_id': 'req-1fdbcaf2-861e-4359-87e5-1d397ebc742a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:17.998477 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1013/2077] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:17.999549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.001418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.013443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9848d27b0c3f4d02b827cce6050866ca', 'request_id': 'req-1fdbcaf2-861e-4359-87e5-1d397ebc742a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.070610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5cc67-0657-4059-b4df-7fb229c3251b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.119796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] The token cache is being invalidate because project 9848d27b0c3f4d02b827cce6050866ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:18.120085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.120892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.127039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1065/2078] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:17 2022] DELETE /identity/v3/projects/9848d27b0c3f4d02b827cce6050866ca => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:18.127039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.127039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.127365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.128884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.129221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.129550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] PATH_INFO: `/v3/projects/1c6b8df0f2664038bf94b7937acc1ff4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.130821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c6b8df0f2664038bf94b7937acc1ff4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.132848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.182567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbcaf2-861e-4359-87e5-1d397ebc742a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.186249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb32efb-c642-478b-ade7-cd1afca56d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.186890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb32efb-c642-478b-ade7-cd1afca56d0b service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.187171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb32efb-c642-478b-ade7-cd1afca56d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.187446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb32efb-c642-478b-ade7-cd1afca56d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.189211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cdb32efb-c642-478b-ade7-cd1afca56d0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.249072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c6b8df0f2664038bf94b7937acc1ff4', 'request_id': 'req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.261699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb32efb-c642-478b-ade7-cd1afca56d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.263094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb32efb-c642-478b-ade7-cd1afca56d0b service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.273037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1066/2079] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:18.274338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc262a3-b39b-4ab1-ae40-d4224167fe2c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.274630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc262a3-b39b-4ab1-ae40-d4224167fe2c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.274907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc262a3-b39b-4ab1-ae40-d4224167fe2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.294008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9bc262a3-b39b-4ab1-ae40-d4224167fe2c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:18.295811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9bc262a3-b39b-4ab1-ae40-d4224167fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.342622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c6b8df0f2664038bf94b7937acc1ff4', 'request_id': 'req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.349726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c6b8df0f2664038bf94b7937acc1ff4', 'request_id': 'req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.364571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] The token cache is being invalidate because project 1c6b8df0f2664038bf94b7937acc1ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:18.364571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.365216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.367195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1014/2080] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:17 2022] DELETE /identity/v3/projects/1c6b8df0f2664038bf94b7937acc1ff4 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:18.368211 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.374816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.388461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04cc7e-6c57-4b51-bbd6-e3b8257264df admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.390691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1067/2081] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:18.392255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807bfe5-40e8-47ae-a2c4-14d911fa66b0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.392612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807bfe5-40e8-47ae-a2c4-14d911fa66b0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.392947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807bfe5-40e8-47ae-a2c4-14d911fa66b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.400817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.401810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.402171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.402459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] PATH_INFO: `/v3/users/f3dc7683c04d42ffa5642c97a2f25d1d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.403685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dc7683c04d42ffa5642c97a2f25d1d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.405856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.416986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6807bfe5-40e8-47ae-a2c4-14d911fa66b0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:18.418311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6807bfe5-40e8-47ae-a2c4-14d911fa66b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.426924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1068/2082] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:18.428261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b47f7-6a25-4789-b4e9-c4e840b19741 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.428608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b47f7-6a25-4789-b4e9-c4e840b19741 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.428944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b47f7-6a25-4789-b4e9-c4e840b19741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.448855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f26b47f7-6a25-4789-b4e9-c4e840b19741 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:18.450023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f26b47f7-6a25-4789-b4e9-c4e840b19741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.457785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1069/2083] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:18.458946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20d8a-93e5-4fe8-9c6d-0a3c232b5e46 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.459231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20d8a-93e5-4fe8-9c6d-0a3c232b5e46 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.459505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20d8a-93e5-4fe8-9c6d-0a3c232b5e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.477225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e5b20d8a-93e5-4fe8-9c6d-0a3c232b5e46 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:18.478359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5b20d8a-93e5-4fe8-9c6d-0a3c232b5e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.486204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1070/2084] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:18.487020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20d8a-93e5-4fe8-9c6d-0a3c232b5e46 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.497005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20d8a-93e5-4fe8-9c6d-0a3c232b5e46 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.502592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.503503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.503503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.503801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.504525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:18.504824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.506411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.529900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1071/2085] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:18.529900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.529900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026119a4-9fd1-4dca-ba76-bcb811c41a8c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.536129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.536769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.537052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.537323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.538035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:18.538337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.539945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.553324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1072/2086] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:18.554177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.563526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377affdf-018b-4eb6-9ce2-421dbb94c793 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.568192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.568829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.569109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.569381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.570055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.571978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.620018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dc7683c04d42ffa5642c97a2f25d1d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.701295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1073/2087] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:18.702180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.710975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dc7683c04d42ffa5642c97a2f25d1d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.711771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dc7683c04d42ffa5642c97a2f25d1d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.724425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dc7683c04d42ffa5642c97a2f25d1d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.731184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dc7683c04d42ffa5642c97a2f25d1d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:18.741110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1015/2088] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:18 2022] DELETE /identity/v3/users/f3dc7683c04d42ffa5642c97a2f25d1d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:18.741110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.741110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.767956 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ea10d-c8c9-4b64-a78f-5c135e0360c6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.811947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f34b1-ebf3-4a8d-9256-9d1f55206ecb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.816379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.817037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.817323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.817601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.820075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.831709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.832999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.833414 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.833958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.835075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.837622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.890695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.892282 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.903371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1016/2089] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:18.904206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.905499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:18.919513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b51b4-4e44-41e1-89b5-032d9ab18b3f service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:18.924425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:18.925166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:18.925453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:18.925723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:18.926413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:18.928389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:18.971351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1074/2090] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/projects => generated 391 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:18.977027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:18.978727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.048454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4022c3-6768-44f5-b53e-3139fac71c70 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.053717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.054372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.054663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.054941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.055650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.057600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.077823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1017/2091] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:19.080261 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.081500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.096151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814e62d-e0a8-4a20-a419-5480c1edb81a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.101380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.102432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.102778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.103171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] PATH_INFO: `/v3/projects/494afadf76c94f859b141309a6a3e37b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.104557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=494afadf76c94f859b141309a6a3e37b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.106935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.223225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1075/2092] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:18 2022] POST /identity/v3/users => generated 296 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:19.224082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.235284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101390d-2c06-40c6-aaff-f961ed6dc306 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.240485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.241132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.241412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.241686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.242413 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.244043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.254882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1076/2093] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:19.255874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.258598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494afadf76c94f859b141309a6a3e37b', 'request_id': 'req-a478091e-c42b-4895-9c80-e64e44f296ac'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:19.265256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c3f4-aa16-4e18-9ef4-08fb5eab71c5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.325597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494afadf76c94f859b141309a6a3e37b', 'request_id': 'req-a478091e-c42b-4895-9c80-e64e44f296ac'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:19.330210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.331538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.331538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.332089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494afadf76c94f859b141309a6a3e37b', 'request_id': 'req-a478091e-c42b-4895-9c80-e64e44f296ac'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:19.332206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.332840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.334560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.343958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] The token cache is being invalidate because project 494afadf76c94f859b141309a6a3e37b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:19.347410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:19.347526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1077/2094] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:19.347526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.347703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:19.347953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1018/2095] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:19 2022] DELETE /identity/v3/projects/494afadf76c94f859b141309a6a3e37b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:19.348714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.348838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.349994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.361586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bd866-df30-4a8f-804d-08424daf7e7c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.363101 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478091e-c42b-4895-9c80-e64e44f296ac admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.366952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.367610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.367734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d315a8-34f5-471d-84e0-fd420bb3fd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.367977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.368283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.368398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d315a8-34f5-471d-84e0-fd420bb3fd50 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.368621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d315a8-34f5-471d-84e0-fd420bb3fd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.368894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d315a8-34f5-471d-84e0-fd420bb3fd50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.369015 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.369674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d315a8-34f5-471d-84e0-fd420bb3fd50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.370731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.371541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d315a8-34f5-471d-84e0-fd420bb3fd50 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.385397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1019/2096] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:19.385397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4f025-9a75-4e43-868b-ecd92b86d13b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.385397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4f025-9a75-4e43-868b-ecd92b86d13b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.385397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4f025-9a75-4e43-868b-ecd92b86d13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.395179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1078/2097] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:19.395179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.402045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5670ba-d9ce-44d3-ade9-0a18497dcda6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.407752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b8b4f025-9a75-4e43-868b-ecd92b86d13b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:19.410306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b8b4f025-9a75-4e43-868b-ecd92b86d13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.412323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eeb77f-d96a-4d29-88ac-684a2e3e1eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.413614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeb77f-d96a-4d29-88ac-684a2e3e1eca admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.414372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeb77f-d96a-4d29-88ac-684a2e3e1eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.414649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeb77f-d96a-4d29-88ac-684a2e3e1eca admin admin] PATH_INFO: `/v3/projects/6544668e69b64382b67b4ec8355005e7/users/3018834260554835b5cdbe3d6f68f494/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.419765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1020/2098] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:19.420580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4f025-9a75-4e43-868b-ecd92b86d13b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.432890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4f025-9a75-4e43-868b-ecd92b86d13b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.438644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eeb77f-d96a-4d29-88ac-684a2e3e1eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6544668e69b64382b67b4ec8355005e7, user_id=3018834260554835b5cdbe3d6f68f494, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.439533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e21f08-240e-4751-8798-f4dab6baf96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.441608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e21f08-240e-4751-8798-f4dab6baf96f admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.442095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e21f08-240e-4751-8798-f4dab6baf96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.442587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e21f08-240e-4751-8798-f4dab6baf96f admin admin] PATH_INFO: `/v3/projects/bc305175091a4722866ad691d7ea3add/users/0e3305028a3744d3a160da912ac2daba/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.442981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eeb77f-d96a-4d29-88ac-684a2e3e1eca admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.458695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e21f08-240e-4751-8798-f4dab6baf96f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc305175091a4722866ad691d7ea3add, user_id=0e3305028a3744d3a160da912ac2daba, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.461554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e21f08-240e-4751-8798-f4dab6baf96f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.557354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1079/2099] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:19 2022] PUT /identity/v3/projects/6544668e69b64382b67b4ec8355005e7/users/3018834260554835b5cdbe3d6f68f494/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:19.560310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f709df-48f2-4ba9-92da-5ad637545374 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.560604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f709df-48f2-4ba9-92da-5ad637545374 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.561092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f709df-48f2-4ba9-92da-5ad637545374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.576535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1021/2100] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:19 2022] PUT /identity/v3/projects/bc305175091a4722866ad691d7ea3add/users/0e3305028a3744d3a160da912ac2daba/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:19.578012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e03f68-b97c-4fd1-89a9-86d80ab4996b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.578297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e03f68-b97c-4fd1-89a9-86d80ab4996b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.578748 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e03f68-b97c-4fd1-89a9-86d80ab4996b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.584311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c7f709df-48f2-4ba9-92da-5ad637545374 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:19.585607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c7f709df-48f2-4ba9-92da-5ad637545374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.605940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-67e03f68-b97c-4fd1-89a9-86d80ab4996b None None] MFA Rules not processed for user `3018834260554835b5cdbe3d6f68f494`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:19.607135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-67e03f68-b97c-4fd1-89a9-86d80ab4996b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.646276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1080/2101] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:19.648120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d847dff-244b-486c-886e-e56faaafb14c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.648534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d847dff-244b-486c-886e-e56faaafb14c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.648654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d847dff-244b-486c-886e-e56faaafb14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.676373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9d847dff-244b-486c-886e-e56faaafb14c None None] MFA Rules not processed for user `0e3305028a3744d3a160da912ac2daba`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:19.677589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9d847dff-244b-486c-886e-e56faaafb14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.679542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1022/2102] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:19.679542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e03f68-b97c-4fd1-89a9-86d80ab4996b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.690425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e03f68-b97c-4fd1-89a9-86d80ab4996b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.696356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.697081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.701164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.701164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.701164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:19.701164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.701164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.724972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1023/2103] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:19.724972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.726440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:19.756671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1081/2104] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:19.756671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d847dff-244b-486c-886e-e56faaafb14c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.764766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d847dff-244b-486c-886e-e56faaafb14c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.770528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.771353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.772024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.772406 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.773202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.775379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.792637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12396ee6-b260-46ac-9730-a38f6ab357c3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.796479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.797171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.797474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.798141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.807358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.808820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.818200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1024/2105] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:19.818975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.829536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567753f-f19d-4ba4-b675-03a7d1c5aec3 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.834647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.835353 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.835656 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.835948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.836700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.838575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.873374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1082/2106] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:19.877573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:19.941878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd8638-1446-46f9-831b-50425d0d119d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:19.946626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:19.947298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:19.947578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:19.947889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:19.948576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:19.950495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:19.962557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1025/2107] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:19.966662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.029161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de223d1-9a5c-449c-b3de-6e921699e4ef admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.034086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.034793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.035164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.035524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.036307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.038291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.118102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1083/2108] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:20.118311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1026/2109] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:19 2022] POST /identity/v3/users => generated 301 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:20.121614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.121766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.132229 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a2309-a042-47b1-a96d-6802127d0cd2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.132395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde16d12-f0e1-43a5-9c32-85668d098696 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.137015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.137145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.137668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.137765 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.137951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.138045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.138227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.138313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.138997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.139116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.140612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.140708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.154704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1027/2110] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:20.154878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1084/2111] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:20.155680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.155812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.165431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a166-3465-4b49-b731-8abf2c5568cd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.165571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf08cd6-8283-4509-b7c0-bf49d09ce2c5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.173786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.173913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.174016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.174178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.174254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.174331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.174975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.175052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] PATH_INFO: `/v3/projects/83a767d006664a75bca9eaaadc1080d5/users/ccbc5699ce5d4974b1d4db7205c07463/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.175135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.175135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.187661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1028/2112] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:20.189919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.192434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a767d006664a75bca9eaaadc1080d5, user_id=ccbc5699ce5d4974b1d4db7205c07463, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.195243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.201757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e8509-dd3a-4c15-b980-91c0dea634f5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.206722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92824d92-d020-4680-b30d-ad8401e3356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.207824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824d92-d020-4680-b30d-ad8401e3356c admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.208102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824d92-d020-4680-b30d-ad8401e3356c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.208373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824d92-d020-4680-b30d-ad8401e3356c admin admin] PATH_INFO: `/v3/projects/9886c48c5650408cacaa4e7f78586404/users/426aa73007cd4ca3b8c165b87f69c02e/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.224795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92824d92-d020-4680-b30d-ad8401e3356c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9886c48c5650408cacaa4e7f78586404, user_id=426aa73007cd4ca3b8c165b87f69c02e, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.227520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92824d92-d020-4680-b30d-ad8401e3356c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.331429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1085/2113] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:20 2022] PUT /identity/v3/projects/83a767d006664a75bca9eaaadc1080d5/users/ccbc5699ce5d4974b1d4db7205c07463/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:20.335354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.369232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1029/2114] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:20 2022] PUT /identity/v3/projects/9886c48c5650408cacaa4e7f78586404/users/426aa73007cd4ca3b8c165b87f69c02e/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:20.374064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.374351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.375039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.394878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:20.396164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:20.402903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5cd27-896f-4152-94e9-37d588a2ed13 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.453575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1030/2115] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:20.458981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.459686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.460001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.460304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.461036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.462637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.463085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.464539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:20.473172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1086/2116] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:20.477145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.488007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfe171-6a02-41b6-9e29-5863c756b35e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.496646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb1cb3-444c-4788-8d2d-21fd0383b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.496646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1cb3-444c-4788-8d2d-21fd0383b167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.496646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1cb3-444c-4788-8d2d-21fd0383b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.496646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1cb3-444c-4788-8d2d-21fd0383b167 admin admin] PATH_INFO: `/v3/projects/83a767d006664a75bca9eaaadc1080d5/users/ccbc5699ce5d4974b1d4db7205c07463/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.497246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb1cb3-444c-4788-8d2d-21fd0383b167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a767d006664a75bca9eaaadc1080d5, user_id=ccbc5699ce5d4974b1d4db7205c07463, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.499952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb1cb3-444c-4788-8d2d-21fd0383b167 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.511890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1087/2117] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:20 2022] PUT /identity/v3/projects/83a767d006664a75bca9eaaadc1080d5/users/ccbc5699ce5d4974b1d4db7205c07463/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:20.522483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccea37-5acb-4370-b051-9c23130cfd1f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.522775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccea37-5acb-4370-b051-9c23130cfd1f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.525274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccea37-5acb-4370-b051-9c23130cfd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.530188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc901a-87d0-46d4-8c3b-b258b1bb99b3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.555273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-13ccea37-5acb-4370-b051-9c23130cfd1f None None] MFA Rules not processed for user `ccbc5699ce5d4974b1d4db7205c07463`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:20.556740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-13ccea37-5acb-4370-b051-9c23130cfd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:20.582908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.584062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.584062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.584221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.650705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1088/2118] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:20.663891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccea37-5acb-4370-b051-9c23130cfd1f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:20.671433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.679859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.686186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1031/2119] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:16:20.744593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccea37-5acb-4370-b051-9c23130cfd1f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:20.750147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb357-a31a-4da7-96e3-43d28bb49645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:20.751257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb357-a31a-4da7-96e3-43d28bb49645 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:20.751650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb357-a31a-4da7-96e3-43d28bb49645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:20.752024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb357-a31a-4da7-96e3-43d28bb49645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:20.768565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eb357-a31a-4da7-96e3-43d28bb49645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:20.771418 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eb357-a31a-4da7-96e3-43d28bb49645 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:20.791992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1089/2120] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:16:21.273354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:21.287465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6c80-54cc-4a4d-b585-d60a629658a5 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:21.298194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:21.298430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:21.298952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:21.299280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] PATH_INFO: `/v3/users/0e3305028a3744d3a160da912ac2daba` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:21.314423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3305028a3744d3a160da912ac2daba)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:21.317221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:21.550541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3305028a3744d3a160da912ac2daba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:21.648609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3305028a3744d3a160da912ac2daba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:21.649273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3305028a3744d3a160da912ac2daba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:21.662705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3305028a3744d3a160da912ac2daba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:21.669949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3305028a3744d3a160da912ac2daba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:21.681998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1032/2121] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:21 2022] DELETE /identity/v3/users/0e3305028a3744d3a160da912ac2daba => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:21.685988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:21.686385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:21.686757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:21.719331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:21.723889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:21.802486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1090/2122] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:21.812079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:21.813538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:21.873338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6853760-c7aa-45c6-9af9-6571086dc43e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:21.877159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:21.877887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:21.878292 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:21.878638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:21.888841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:21.890294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:21.901109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1033/2123] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:22.429905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:22.432068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:22.476669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db435aa-cc60-4738-be80-a6e29a0d12dd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:22.483491 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:22.484446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:22.484730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:22.488556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] PATH_INFO: `/v3/projects/bc305175091a4722866ad691d7ea3add` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:22.488556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc305175091a4722866ad691d7ea3add)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:22.491989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:22.613668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc305175091a4722866ad691d7ea3add', 'request_id': 'req-c1f605e4-7367-4576-b315-448d50880080'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:22.623826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc305175091a4722866ad691d7ea3add', 'request_id': 'req-c1f605e4-7367-4576-b315-448d50880080'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:22.632429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc305175091a4722866ad691d7ea3add', 'request_id': 'req-c1f605e4-7367-4576-b315-448d50880080'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:22.715281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] The token cache is being invalidate because project bc305175091a4722866ad691d7ea3add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:22.716952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:22.718346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:22.722359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1091/2124] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:22 2022] DELETE /identity/v3/projects/bc305175091a4722866ad691d7ea3add => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:22.731601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:22.744703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b4da3-ff56-451f-91d9-33972aa42cf9 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:22.795211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:22.797059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:22.813454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:22.814325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:22.814604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:22.814869 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] PATH_INFO: `/v3/users/ccbc5699ce5d4974b1d4db7205c07463` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:22.816705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccbc5699ce5d4974b1d4db7205c07463)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:22.818626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:22.871274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f605e4-7367-4576-b315-448d50880080 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:22.876331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:22.877214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:22.878433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:22.879148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] PATH_INFO: `/v3/users/ee67269a94d6494597cdf27b19c2f371` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:22.880597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee67269a94d6494597cdf27b19c2f371)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:22.882564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:23.024678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccbc5699ce5d4974b1d4db7205c07463'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.066084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee67269a94d6494597cdf27b19c2f371'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.155768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccbc5699ce5d4974b1d4db7205c07463'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.156622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccbc5699ce5d4974b1d4db7205c07463'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.183300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccbc5699ce5d4974b1d4db7205c07463'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.191165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4fa30fde-28c5-4042-b14d-4ffd440ca304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccbc5699ce5d4974b1d4db7205c07463'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.198165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1034/2125] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:22 2022] DELETE /identity/v3/users/ccbc5699ce5d4974b1d4db7205c07463 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:23.226156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee67269a94d6494597cdf27b19c2f371'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.239855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee67269a94d6494597cdf27b19c2f371'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.241047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee67269a94d6494597cdf27b19c2f371'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.263789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee67269a94d6494597cdf27b19c2f371'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.279117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1092/2126] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:22 2022] DELETE /identity/v3/users/ee67269a94d6494597cdf27b19c2f371 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:23.283622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:23.285453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:23.285914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:23.319436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:23.320813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:23.383112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1035/2127] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:23.392576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:23.394224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:23.491924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63b61-3bea-44f4-964a-b4f8e334e5f1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:23.495797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763380c-896f-4e5f-8f52-e01b7b410c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:23.496459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763380c-896f-4e5f-8f52-e01b7b410c20 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:23.496780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763380c-896f-4e5f-8f52-e01b7b410c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:23.497145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763380c-896f-4e5f-8f52-e01b7b410c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:23.506831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763380c-896f-4e5f-8f52-e01b7b410c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:23.508333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763380c-896f-4e5f-8f52-e01b7b410c20 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:23.520678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1093/2128] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:23.693814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:23.695234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:23.707484 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e814b-64d2-4340-a170-47dfd4df86c6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:23.720143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:23.722158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:23.722158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:23.722453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] PATH_INFO: `/v3/projects/83a767d006664a75bca9eaaadc1080d5` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:23.724719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a767d006664a75bca9eaaadc1080d5)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:23.726673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:23.882505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a767d006664a75bca9eaaadc1080d5', 'request_id': 'req-f4c0bd97-d584-46ea-a15f-fd5c60534a81'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.964916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a767d006664a75bca9eaaadc1080d5', 'request_id': 'req-f4c0bd97-d584-46ea-a15f-fd5c60534a81'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:23.979569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a767d006664a75bca9eaaadc1080d5', 'request_id': 'req-f4c0bd97-d584-46ea-a15f-fd5c60534a81'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.015680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] The token cache is being invalidate because project 83a767d006664a75bca9eaaadc1080d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:24.015680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.015680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f4c0bd97-d584-46ea-a15f-fd5c60534a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.019493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1036/2129] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:23 2022] DELETE /identity/v3/projects/83a767d006664a75bca9eaaadc1080d5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:24.024929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74d488-c3d6-44ce-b620-b31814266bda None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.025763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74d488-c3d6-44ce-b620-b31814266bda None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.026394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74d488-c3d6-44ce-b620-b31814266bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.049802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f74d488-c3d6-44ce-b620-b31814266bda None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:24.051360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f74d488-c3d6-44ce-b620-b31814266bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:24.113019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1094/2130] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:24.121666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.122029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.122378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.141588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:24.142802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:24.152565 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1037/2131] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:24.160688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74d488-c3d6-44ce-b620-b31814266bda None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:24.165195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:24.167800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:24.181249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74d488-c3d6-44ce-b620-b31814266bda None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:24.185134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:24.186136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.186958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.187317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.188245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:24.188774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:24.191434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:24.204605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b18f-97fd-44c0-be6a-d5dfede76bcd None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:24.206379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1095/2132] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:24.210345 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:24.215099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:24.215938 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.216218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.216491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] PATH_INFO: `/v3/projects/a5799c76cf744fb2a1f3d73cf6f70c3e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.220366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5799c76cf744fb2a1f3d73cf6f70c3e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:24.220693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1289fd-6bce-46b8-871c-4588033d3aba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:24.222410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:24.231343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:24.232226 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.232782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.233129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.234821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:24.238707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:24.381509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5799c76cf744fb2a1f3d73cf6f70c3e', 'request_id': 'req-1666d3ec-115f-40d3-87e7-32f61905478a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.411929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1096/2133] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:24 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:24.420929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:24.477614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5799c76cf744fb2a1f3d73cf6f70c3e', 'request_id': 'req-1666d3ec-115f-40d3-87e7-32f61905478a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.490936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5799c76cf744fb2a1f3d73cf6f70c3e', 'request_id': 'req-1666d3ec-115f-40d3-87e7-32f61905478a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.497877 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9e2a6-c5f1-43ea-b6e8-e608d46062a3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:24.504463 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:24.506048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.506504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.506965 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.508081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:24.509904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] The token cache is being invalidate because project a5799c76cf744fb2a1f3d73cf6f70c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:24.510255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.510976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:24.511093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:24.524444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1038/2134] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:24 2022] DELETE /identity/v3/projects/a5799c76cf744fb2a1f3d73cf6f70c3e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:24.635140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1097/2135] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:24 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:24.636054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:24.637549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:24.733094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666d3ec-115f-40d3-87e7-32f61905478a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:24.738862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:24.740154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.740450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.740753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.741503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:24.743299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:24.759231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1039/2136] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:24.760132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:24.772979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9f48-fc52-478f-a931-202dc16c7baf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:24.779392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a135e-1806-4e80-9148-b236e22f8ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:24.780114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a135e-1806-4e80-9148-b236e22f8ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.780709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a135e-1806-4e80-9148-b236e22f8ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.781006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a135e-1806-4e80-9148-b236e22f8ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.781747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a135e-1806-4e80-9148-b236e22f8ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:24.783394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a135e-1806-4e80-9148-b236e22f8ab9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:24.794601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1098/2137] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:24.799189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:24.812817 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635817f-23b4-416d-97dc-7a0a2c4a4464 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:24.818366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:24.819274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.819558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.820038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] PATH_INFO: `/v3/projects/2268806bbb1a4fc986186dea854c1fed/users/8ca3b77c9f85450c96cf688711870273/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.838485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2268806bbb1a4fc986186dea854c1fed, user_id=8ca3b77c9f85450c96cf688711870273, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:24.845004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:24.912156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1040/2138] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:24 2022] PUT /identity/v3/projects/2268806bbb1a4fc986186dea854c1fed/users/8ca3b77c9f85450c96cf688711870273/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:24.917620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaec90-7cb1-42cd-b399-65dc9bbf5a6a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:24.917915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaec90-7cb1-42cd-b399-65dc9bbf5a6a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:24.918211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eaec90-7cb1-42cd-b399-65dc9bbf5a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:24.938758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-05eaec90-7cb1-42cd-b399-65dc9bbf5a6a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:24.940712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-05eaec90-7cb1-42cd-b399-65dc9bbf5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:25.008310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1099/2139] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:25.017836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:25.019873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:25.080049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939429-4f9f-4b7b-be5d-c9c1a2813028 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:25.084093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:25.084749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:25.085028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:25.085302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:25.098788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:25.102329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:25.125203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1041/2140] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:16:26.716373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55e48c-8bf5-4df9-ada7-1847910a6440 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:26.716827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55e48c-8bf5-4df9-ada7-1847910a6440 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:26.717097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55e48c-8bf5-4df9-ada7-1847910a6440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:26.776942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8a55e48c-8bf5-4df9-ada7-1847910a6440 None None] MFA Rules not processed for user `8ca3b77c9f85450c96cf688711870273`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:26.778346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8a55e48c-8bf5-4df9-ada7-1847910a6440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:26.895779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1100/2141] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:26.900512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:26.916373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a2d9-5d73-4ebf-89b8-da601b98b536 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:26.918005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a998a-7168-4f13-a657-e6b599de78b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:26.918677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a998a-7168-4f13-a657-e6b599de78b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:26.918987 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a998a-7168-4f13-a657-e6b599de78b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:26.919225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a998a-7168-4f13-a657-e6b599de78b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:26.920010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a998a-7168-4f13-a657-e6b599de78b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:26.921931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a998a-7168-4f13-a657-e6b599de78b4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:27.059483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1042/2142] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:26 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:27.064171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55e48c-8bf5-4df9-ada7-1847910a6440 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:27.076725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55e48c-8bf5-4df9-ada7-1847910a6440 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:27.165822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:27.166576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:27.166871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:27.167154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:27.168247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:27.170224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:27.250909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:27.251198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:27.251474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:27.278491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] MFA Rules not processed for user `426aa73007cd4ca3b8c165b87f69c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:27.279743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:27.280185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1101/2143] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:27 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:27.284107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:27.294284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394785b6-8d07-4b0b-be40-96b917475246 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:27.299096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:27.300562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:27.301285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:27.301285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:27.301999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:27.303600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:27.317514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1102/2144] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:27.321887 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:27.339351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc25ea5-a6e2-48a3-866a-e489bd7a19e5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:27.344975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:27.345881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:27.346686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:27.346978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] PATH_INFO: `/v3/projects/2e2de37c50f948e381822287eed1f7b3/users/87c4d5aad8744f2caaf3f458c9d33a7f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:27.368526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2de37c50f948e381822287eed1f7b3, user_id=87c4d5aad8744f2caaf3f458c9d33a7f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:27.378935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:27.379131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1043/2145] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:27.394262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:27.395816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:27.464560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1103/2146] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:27 2022] PUT /identity/v3/projects/2e2de37c50f948e381822287eed1f7b3/users/87c4d5aad8744f2caaf3f458c9d33a7f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:27.469157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:27.469634 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f76dd-e9e3-491f-b5d3-81154634a4e2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:27.537072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eb926-c0d3-4702-8b51-880e05bc1efb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:27.540483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:27.541741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:27.542246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:27.542357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:27.542784 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:27.543351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:27.543730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:27.544093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:27.544893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:27.546573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:27.558274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1104/2147] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:27.562434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:27.572349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc7c7-0317-41ff-b252-b679ee5ab951 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:27.576938 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce96008-face-42b3-b7fc-23af5fb556e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:27.577795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce96008-face-42b3-b7fc-23af5fb556e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:27.578075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce96008-face-42b3-b7fc-23af5fb556e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:27.578350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce96008-face-42b3-b7fc-23af5fb556e6 admin admin] PATH_INFO: `/v3/projects/2e2de37c50f948e381822287eed1f7b3/users/87c4d5aad8744f2caaf3f458c9d33a7f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:27.580335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce96008-face-42b3-b7fc-23af5fb556e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2de37c50f948e381822287eed1f7b3, user_id=87c4d5aad8744f2caaf3f458c9d33a7f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:27.584079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce96008-face-42b3-b7fc-23af5fb556e6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:27.598852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1105/2148] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:27 2022] PUT /identity/v3/projects/2e2de37c50f948e381822287eed1f7b3/users/87c4d5aad8744f2caaf3f458c9d33a7f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:27.608547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:27.610163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:27.620859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1044/2149] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:16:29.345914 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6559a-a2a8-4906-a2c4-4796e3d1cc01 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:29.346683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6559a-a2a8-4906-a2c4-4796e3d1cc01 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:29.346683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6559a-a2a8-4906-a2c4-4796e3d1cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:29.404369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9de6559a-a2a8-4906-a2c4-4796e3d1cc01 None None] MFA Rules not processed for user `87c4d5aad8744f2caaf3f458c9d33a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:29.408377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9de6559a-a2a8-4906-a2c4-4796e3d1cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:29.536398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1106/2150] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:29.548564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:29.550579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:29.619235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c697057-962f-453a-b072-3afd6f7a39b4 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:29.672887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:29.673554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:29.673837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:29.674108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:29.749550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:29.751179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:29.761044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1045/2151] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:30.427568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:30.428068 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:30.428159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:30.446534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] MFA Rules not processed for user `426aa73007cd4ca3b8c165b87f69c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:30.448649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:30.515196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1107/2152] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:31.575493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:31.576994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:31.588558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf54056-72e8-4e79-8a97-40fe9bd27dc1 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:31.593381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a83b8-a3d8-49cb-af26-1b0ab45822f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:31.594121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a83b8-a3d8-49cb-af26-1b0ab45822f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:31.595685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a83b8-a3d8-49cb-af26-1b0ab45822f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:31.595685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a83b8-a3d8-49cb-af26-1b0ab45822f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:31.598652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:31.600140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:31.606848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a83b8-a3d8-49cb-af26-1b0ab45822f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:31.609674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a83b8-a3d8-49cb-af26-1b0ab45822f8 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:31.623558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1046/2153] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:31.664806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b6710-0e96-405c-9225-9c8925fdddbb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:31.669212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:31.670124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:31.670401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:31.670672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] PATH_INFO: `/v3/users/2272548a065f45cf877e51be78d0d471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:31.671898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=2272548a065f45cf877e51be78d0d471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:31.673821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:31.888248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2272548a065f45cf877e51be78d0d471'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:31.973891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2272548a065f45cf877e51be78d0d471'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:31.982182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2272548a065f45cf877e51be78d0d471'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:31.983058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2272548a065f45cf877e51be78d0d471'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:31.997000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2272548a065f45cf877e51be78d0d471'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:32.009131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1108/2154] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:31 2022] DELETE /identity/v3/users/2272548a065f45cf877e51be78d0d471 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:32.014520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:32.014815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:32.015122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:32.036457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:32.036457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:32.093355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1047/2155] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:32.105482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:32.106878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:32.163047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff501786-2789-495b-9a5e-84ed20ade8be admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:32.166640 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:32.167311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:32.167593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:32.167890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:32.177435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:32.178862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:32.188061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1109/2156] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:32.604314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:32.605754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:32.621555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bddd75-8b34-4c2a-bc03-317b0f0bba6f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:32.626280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:32.627139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:32.627421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:32.627751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] PATH_INFO: `/v3/projects/cc1e9a564da046168038188d4f932720` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:32.629001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1e9a564da046168038188d4f932720)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:32.631166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:32.733496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1e9a564da046168038188d4f932720', 'request_id': 'req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:32.793739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1e9a564da046168038188d4f932720', 'request_id': 'req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:32.799243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1e9a564da046168038188d4f932720', 'request_id': 'req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:32.811270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] The token cache is being invalidate because project cc1e9a564da046168038188d4f932720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:32.811549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:32.812196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:32.813752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1048/2157] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:32 2022] DELETE /identity/v3/projects/cc1e9a564da046168038188d4f932720 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:32.817745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:32.827417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0bf2-26c9-421e-a2ef-8b379c01deae service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:32.882658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:32.883541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:32.883878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:32.884159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] PATH_INFO: `/v3/users/322cb0d6ded44c2aaa81a695e75da922` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:32.885342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322cb0d6ded44c2aaa81a695e75da922)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:32.887267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:33.070443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322cb0d6ded44c2aaa81a695e75da922'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.139388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322cb0d6ded44c2aaa81a695e75da922'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.148512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322cb0d6ded44c2aaa81a695e75da922'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.165967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322cb0d6ded44c2aaa81a695e75da922'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.165967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e5c816c-529f-4a46-9d92-8d4f771030b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322cb0d6ded44c2aaa81a695e75da922'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.168672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1110/2158] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:32 2022] DELETE /identity/v3/users/322cb0d6ded44c2aaa81a695e75da922 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:33.573170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:33.574666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:33.592727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55af8c0-1827-4d64-b4b4-7c244b9cdd46 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:33.665830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:33.665830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:33.665830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:33.665830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] PATH_INFO: `/v3/projects/2dcf5e7a61384badb2f10b4d053cd9e2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:33.666784 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dcf5e7a61384badb2f10b4d053cd9e2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:33.668773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:33.804505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dcf5e7a61384badb2f10b4d053cd9e2', 'request_id': 'req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.868319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dcf5e7a61384badb2f10b4d053cd9e2', 'request_id': 'req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.874193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dcf5e7a61384badb2f10b4d053cd9e2', 'request_id': 'req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.884980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] The token cache is being invalidate because project 2dcf5e7a61384badb2f10b4d053cd9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:33.885264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.885890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a7bee4e9-8a8a-4c67-8d16-56ee17ce8550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:33.887440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1049/2159] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:33 2022] DELETE /identity/v3/projects/2dcf5e7a61384badb2f10b4d053cd9e2 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:33.893919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076cdc3-8a4b-4718-8d73-0638bc729fee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:33.894389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076cdc3-8a4b-4718-8d73-0638bc729fee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:33.894477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076cdc3-8a4b-4718-8d73-0638bc729fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:33.912420 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9076cdc3-8a4b-4718-8d73-0638bc729fee None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:33.913599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9076cdc3-8a4b-4718-8d73-0638bc729fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:33.972565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1111/2160] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:33.980963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f96a1c-f309-41d4-9855-116b5f36cb97 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:33.981256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f96a1c-f309-41d4-9855-116b5f36cb97 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:33.981538 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f96a1c-f309-41d4-9855-116b5f36cb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:33.999546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-90f96a1c-f309-41d4-9855-116b5f36cb97 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:34.000770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-90f96a1c-f309-41d4-9855-116b5f36cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:34.009088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1050/2161] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:34.013307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076cdc3-8a4b-4718-8d73-0638bc729fee None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:34.024202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076cdc3-8a4b-4718-8d73-0638bc729fee None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:34.028716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:34.029366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.029649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.029922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.030646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:34.030948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:34.032573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:34.044618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1112/2162] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:34.048907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f96a1c-f309-41d4-9855-116b5f36cb97 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:34.058313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f96a1c-f309-41d4-9855-116b5f36cb97 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:34.062801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:34.063448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.063759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.064034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.064729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:34.067511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:34.169320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1051/2163] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:34 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:34.170760 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:34.181791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b69bd-8218-4319-b7e7-469e21b55ee0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:34.240100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:34.240793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.241078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.241357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.242062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:34.244026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:34.352639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1113/2164] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:34 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:34.356149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:34.366846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dfa79-2995-450f-8aeb-366461a85fa2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:34.371436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:34.372128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.372412 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.372692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.373411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:34.374996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:34.385171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1052/2165] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:34.389022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:34.399475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f553e-777b-4cc1-aee8-b7a24da886ca admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:34.404102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f59495-576f-4473-bc90-fa0a707ab4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:34.404751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f59495-576f-4473-bc90-fa0a707ab4ee admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.405032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f59495-576f-4473-bc90-fa0a707ab4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.405307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f59495-576f-4473-bc90-fa0a707ab4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.406028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f59495-576f-4473-bc90-fa0a707ab4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:34.407606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f59495-576f-4473-bc90-fa0a707ab4ee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:34.418093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1114/2166] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:34.421926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:34.432143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49e0e1-7de1-4619-8c46-e4cde5076c45 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:34.436674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:34.437533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.437815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.438089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] PATH_INFO: `/v3/projects/147da5d5de00409eb9b982affecf976b/users/7a3a047e0a1342ce91a8889f6b6c0b1e/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.452271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147da5d5de00409eb9b982affecf976b, user_id=7a3a047e0a1342ce91a8889f6b6c0b1e, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:34.454918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:34.556866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1053/2167] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:34 2022] PUT /identity/v3/projects/147da5d5de00409eb9b982affecf976b/users/7a3a047e0a1342ce91a8889f6b6c0b1e/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:34.569209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.569583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.569929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.598006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] MFA Rules not processed for user `7a3a047e0a1342ce91a8889f6b6c0b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:34.599209 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:34.665830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1115/2168] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:34.677260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:34.679028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:34.734098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180029e9-4fe6-4177-94d6-97357c29b0a0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:34.738694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f338fc-228e-4f16-911f-de7f7accfccf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:34.739335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f338fc-228e-4f16-911f-de7f7accfccf service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:34.739614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f338fc-228e-4f16-911f-de7f7accfccf service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:34.739911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f338fc-228e-4f16-911f-de7f7accfccf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:34.748187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f338fc-228e-4f16-911f-de7f7accfccf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:34.749610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f338fc-228e-4f16-911f-de7f7accfccf service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:34.758753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1054/2169] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:42.417022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:42.419088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:42.558483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ce93c-1a17-4e0b-97d8-7429f103113a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:42.563882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:42.565601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:42.566598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:42.566943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] PATH_INFO: `/v3/users/3018834260554835b5cdbe3d6f68f494` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:42.568327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3018834260554835b5cdbe3d6f68f494)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:42.570233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:42.783518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3018834260554835b5cdbe3d6f68f494'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:42.877224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3018834260554835b5cdbe3d6f68f494'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:42.886731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3018834260554835b5cdbe3d6f68f494'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:42.887555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3018834260554835b5cdbe3d6f68f494'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:42.904546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3018834260554835b5cdbe3d6f68f494'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:42.913500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1116/2170] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:42 2022] DELETE /identity/v3/users/3018834260554835b5cdbe3d6f68f494 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:42.918531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:42.919136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:42.919136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:42.969743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:42.974348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:43.068095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1055/2171] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:43.076109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:43.077554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:43.212132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbd463-a532-44de-8fcc-b5e46f82e9e0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:43.220494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35298476-09e3-4a06-9dca-9171c5d2d671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:43.221748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298476-09e3-4a06-9dca-9171c5d2d671 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:43.222282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298476-09e3-4a06-9dca-9171c5d2d671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:43.222625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298476-09e3-4a06-9dca-9171c5d2d671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:43.232911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35298476-09e3-4a06-9dca-9171c5d2d671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:43.234564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35298476-09e3-4a06-9dca-9171c5d2d671 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:43.251350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1117/2172] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:43.884704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:43.886255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:43.900017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be9f2-9905-4762-9697-066dd99bf02a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:43.904800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:43.905802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:43.906087 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:43.906460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] PATH_INFO: `/v3/projects/6544668e69b64382b67b4ec8355005e7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:43.907841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6544668e69b64382b67b4ec8355005e7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:43.910706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:44.033337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6544668e69b64382b67b4ec8355005e7', 'request_id': 'req-23cfdd61-0bf5-441a-8d28-813e0ebad529'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:44.110904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6544668e69b64382b67b4ec8355005e7', 'request_id': 'req-23cfdd61-0bf5-441a-8d28-813e0ebad529'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:44.118290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6544668e69b64382b67b4ec8355005e7', 'request_id': 'req-23cfdd61-0bf5-441a-8d28-813e0ebad529'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:44.130102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] The token cache is being invalidate because project 6544668e69b64382b67b4ec8355005e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:44.130384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:44.131008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-23cfdd61-0bf5-441a-8d28-813e0ebad529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:44.132625 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1056/2173] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:43 2022] DELETE /identity/v3/projects/6544668e69b64382b67b4ec8355005e7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:44.137590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e29b9-201f-42bb-92c7-4f4da514d00c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.138049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e29b9-201f-42bb-92c7-4f4da514d00c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.138162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e29b9-201f-42bb-92c7-4f4da514d00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.177449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb7e29b9-201f-42bb-92c7-4f4da514d00c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:44.178601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb7e29b9-201f-42bb-92c7-4f4da514d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:44.247540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1118/2174] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:16:44.255924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f078a9b-ec8b-4a64-b8d2-26746098b375 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.256286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f078a9b-ec8b-4a64-b8d2-26746098b375 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.256633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f078a9b-ec8b-4a64-b8d2-26746098b375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.275992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2f078a9b-ec8b-4a64-b8d2-26746098b375 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:44.277255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2f078a9b-ec8b-4a64-b8d2-26746098b375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:44.285566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1057/2175] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:44.290173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e29b9-201f-42bb-92c7-4f4da514d00c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:44.301622 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e29b9-201f-42bb-92c7-4f4da514d00c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:44.306663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:44.307352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.307770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.308046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.308772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:44.309075 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:44.310651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:44.323925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1119/2176] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:44.333186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f078a9b-ec8b-4a64-b8d2-26746098b375 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:44.342700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f078a9b-ec8b-4a64-b8d2-26746098b375 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:44.347406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:44.348093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.348369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.348638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.349336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:44.351227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:44.451620 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1058/2177] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:44 2022] POST /identity/v3/projects => generated 399 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:44.452601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:44.470121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea746185-9a8f-429c-9e34-5e7500d308ee admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:44.553066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:44.553738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.554194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.554555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.555257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:44.557234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:44.666749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1120/2178] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:44 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:44.672765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:44.684392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578fee4-d145-4390-b734-758e08b3f9b6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:44.689283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:44.690006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.690359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.690741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.691546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:44.693551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:44.714912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1059/2179] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:44.716532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:44.728150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13572c3-63be-4e2b-9040-1fe1b1b4de9f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:44.734060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94bfd0-ba2a-4a24-95c0-7c0c97a2650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:44.734060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94bfd0-ba2a-4a24-95c0-7c0c97a2650e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.734375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94bfd0-ba2a-4a24-95c0-7c0c97a2650e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.734803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94bfd0-ba2a-4a24-95c0-7c0c97a2650e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.735581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94bfd0-ba2a-4a24-95c0-7c0c97a2650e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:44.738195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94bfd0-ba2a-4a24-95c0-7c0c97a2650e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:44.752503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1121/2180] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:44.754406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:44.764603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649effd-f5bb-46fa-93d0-fc9b1609ad35 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:44.769487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:44.770467 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.770826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.771109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] PATH_INFO: `/v3/projects/3f63adb32bbc4ca0a6213b8872493564/users/9b73b23523884d26ba883af91dbac978/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.786461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f63adb32bbc4ca0a6213b8872493564, user_id=9b73b23523884d26ba883af91dbac978, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:44.789274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:44.873916 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1060/2181] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:44 2022] PUT /identity/v3/projects/3f63adb32bbc4ca0a6213b8872493564/users/9b73b23523884d26ba883af91dbac978/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:44.884592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:44.884881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:44.885164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:44.912246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] MFA Rules not processed for user `9b73b23523884d26ba883af91dbac978`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:44.913624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:44.983023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1122/2182] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:44.994982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:44.996734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:45.054706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d95037-d52b-4474-a66c-41bc2eca93d9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:45.058642 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:45.059281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:45.059562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:45.059935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:45.069263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:45.070684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:45.079254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1061/2183] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:53.109154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:53.110651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:53.192305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863ba3a-4d6f-46b9-8e15-7b35cd5e7be8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:53.196851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:53.197718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:53.197999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:53.198271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] PATH_INFO: `/v3/users/8ca3b77c9f85450c96cf688711870273` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:53.199461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca3b77c9f85450c96cf688711870273)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:53.201436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:53.357299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca3b77c9f85450c96cf688711870273'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.411087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca3b77c9f85450c96cf688711870273'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.418656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca3b77c9f85450c96cf688711870273'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.419325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca3b77c9f85450c96cf688711870273'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.432094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca3b77c9f85450c96cf688711870273'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.438061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1123/2184] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:53 2022] DELETE /identity/v3/users/8ca3b77c9f85450c96cf688711870273 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:53.672986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:53.685266 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e884eff-f235-4529-adbb-886af17402f4 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:53.752839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:53.753089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:53.753574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:53.753851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] PATH_INFO: `/v3/projects/2268806bbb1a4fc986186dea854c1fed` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:53.755108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2268806bbb1a4fc986186dea854c1fed)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:53.757248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:53.853843 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2268806bbb1a4fc986186dea854c1fed', 'request_id': 'req-c65273e2-4fa6-462c-90ea-207fb77c70e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.924811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2268806bbb1a4fc986186dea854c1fed', 'request_id': 'req-c65273e2-4fa6-462c-90ea-207fb77c70e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.931667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2268806bbb1a4fc986186dea854c1fed', 'request_id': 'req-c65273e2-4fa6-462c-90ea-207fb77c70e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.942920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] The token cache is being invalidate because project 2268806bbb1a4fc986186dea854c1fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:53.943231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.943914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:53.945454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1062/2185] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:53 2022] DELETE /identity/v3/projects/2268806bbb1a4fc986186dea854c1fed => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:53.949398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:53.950830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:53.965808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d3304-cb9d-4391-a89a-062fbd7b1127 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:53.998978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:54.000433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:54.028149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:54.033644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:54.033644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:54.033644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] PATH_INFO: `/v3/users/87c4d5aad8744f2caaf3f458c9d33a7f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:54.033644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c4d5aad8744f2caaf3f458c9d33a7f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:54.033644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:54.066349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65273e2-4fa6-462c-90ea-207fb77c70e8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:54.071208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:54.072118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:54.072398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:54.072667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] PATH_INFO: `/v3/users/9b73b23523884d26ba883af91dbac978` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:54.073880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b73b23523884d26ba883af91dbac978)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:54.075841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:54.244832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c4d5aad8744f2caaf3f458c9d33a7f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.296478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b73b23523884d26ba883af91dbac978'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.339488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c4d5aad8744f2caaf3f458c9d33a7f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.348129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c4d5aad8744f2caaf3f458c9d33a7f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.348901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c4d5aad8744f2caaf3f458c9d33a7f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.361999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f1d20166-a271-4b0d-a48b-a5dc1352f31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c4d5aad8744f2caaf3f458c9d33a7f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.369085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1124/2186] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:53 2022] DELETE /identity/v3/users/87c4d5aad8744f2caaf3f458c9d33a7f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:54.390900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b73b23523884d26ba883af91dbac978'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.392706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b73b23523884d26ba883af91dbac978'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.410515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b73b23523884d26ba883af91dbac978'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.420595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b73b23523884d26ba883af91dbac978'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.427704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1063/2187] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:16:53 2022] DELETE /identity/v3/users/9b73b23523884d26ba883af91dbac978 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:16:54.432794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:54.433360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:54.434292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:54.459357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:54.461258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:54.543674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1125/2188] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:54.551665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:54.553288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:54.606653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:54.608301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:54.617423 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414344cb-a691-4c14-b501-3bbf388dd91d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:54.621312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09823b55-3d22-47c9-9407-081eac883437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:54.622035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09823b55-3d22-47c9-9407-081eac883437 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:54.622375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09823b55-3d22-47c9-9407-081eac883437 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:54.623559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09823b55-3d22-47c9-9407-081eac883437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:54.632552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb783f-28bd-4972-ab68-4443095c4764 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:54.635747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09823b55-3d22-47c9-9407-081eac883437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:54.637959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09823b55-3d22-47c9-9407-081eac883437 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:54.638552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:54.639429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:54.639765 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:54.640043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] PATH_INFO: `/v3/projects/2e2de37c50f948e381822287eed1f7b3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:54.641238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2de37c50f948e381822287eed1f7b3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:54.643156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:54.650315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1064/2189] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:54.776643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2de37c50f948e381822287eed1f7b3', 'request_id': 'req-2d86914a-7149-4f73-997e-5ae7252c372a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.782229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2de37c50f948e381822287eed1f7b3', 'request_id': 'req-2d86914a-7149-4f73-997e-5ae7252c372a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.789197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2de37c50f948e381822287eed1f7b3', 'request_id': 'req-2d86914a-7149-4f73-997e-5ae7252c372a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.952390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] The token cache is being invalidate because project 2e2de37c50f948e381822287eed1f7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:54.952390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.953242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d86914a-7149-4f73-997e-5ae7252c372a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:54.960052 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1126/2190] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:54 2022] DELETE /identity/v3/projects/2e2de37c50f948e381822287eed1f7b3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:54.962978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e7ee1-3ede-4abb-8ca9-835c81f8f438 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:54.963938 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e7ee1-3ede-4abb-8ca9-835c81f8f438 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:54.964369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e7ee1-3ede-4abb-8ca9-835c81f8f438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.001860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f89e7ee1-3ede-4abb-8ca9-835c81f8f438 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:55.003965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f89e7ee1-3ede-4abb-8ca9-835c81f8f438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:55.080928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1065/2191] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:55.084858 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38df3-5e46-4ed9-a548-b5d70086258a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.085146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38df3-5e46-4ed9-a548-b5d70086258a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.085421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38df3-5e46-4ed9-a548-b5d70086258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.111404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-94d38df3-5e46-4ed9-a548-b5d70086258a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:55.113807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-94d38df3-5e46-4ed9-a548-b5d70086258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:55.132815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e7ee1-3ede-4abb-8ca9-835c81f8f438 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:55.134958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1127/2192] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:55.144677 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e7ee1-3ede-4abb-8ca9-835c81f8f438 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:55.153342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:55.154271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.155331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.156224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.157796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:16:55.158256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:55.162400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:55.180717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1066/2193] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:16:55.185852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38df3-5e46-4ed9-a548-b5d70086258a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:55.196678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38df3-5e46-4ed9-a548-b5d70086258a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:55.201300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:55.201951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.202229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.202528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.203569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:55.207247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:55.280993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:55.282716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:55.299281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05948937-0df6-4370-bc72-99c35fc0de54 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:55.304694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:55.305954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.306439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.306782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] PATH_INFO: `/v3/projects/3f63adb32bbc4ca0a6213b8872493564` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.308107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f63adb32bbc4ca0a6213b8872493564)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:55.310639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:55.367410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1128/2194] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:55 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:55.368304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:55.427208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f63adb32bbc4ca0a6213b8872493564', 'request_id': 'req-24bb0c68-bcda-4684-92d0-9f3eb161e373'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:55.440298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf8fdf-bcb8-4661-9e5d-3ccbd12167ff admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:55.497603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:55.498444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.498791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.499132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.499926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:55.501956 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:55.513424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f63adb32bbc4ca0a6213b8872493564', 'request_id': 'req-24bb0c68-bcda-4684-92d0-9f3eb161e373'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:55.519900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f63adb32bbc4ca0a6213b8872493564', 'request_id': 'req-24bb0c68-bcda-4684-92d0-9f3eb161e373'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:55.532000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] The token cache is being invalidate because project 3f63adb32bbc4ca0a6213b8872493564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:16:55.532291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:55.532971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:16:55.534889 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1067/2195] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:16:55 2022] DELETE /identity/v3/projects/3f63adb32bbc4ca0a6213b8872493564 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:55.589768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1129/2196] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:55 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:55.591310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:55.597357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:55.614893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb0c68-bcda-4684-92d0-9f3eb161e373 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:55.620510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:55.621222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.621552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.621835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.622583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:55.624513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:55.642846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1068/2197] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:55.643889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:55.655493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737e24c-d209-4b4d-898b-265e3f8f9de8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:55.660759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90f755-2c9b-473f-a45c-ea822715f4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:55.661452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90f755-2c9b-473f-a45c-ea822715f4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.661733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90f755-2c9b-473f-a45c-ea822715f4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.662009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90f755-2c9b-473f-a45c-ea822715f4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.662967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90f755-2c9b-473f-a45c-ea822715f4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:55.664670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90f755-2c9b-473f-a45c-ea822715f4d0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:55.680388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1130/2198] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:55.681873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:55.695913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbc0d1-cacb-4ca8-b998-251a49c7f0f7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:55.702378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:55.703333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.703596 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.704041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] PATH_INFO: `/v3/projects/a24091b8fe594c3192007255f87f2fb4/users/a2aa2969716948be9aeab78be8220d45/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.722839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24091b8fe594c3192007255f87f2fb4, user_id=a2aa2969716948be9aeab78be8220d45, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:55.726262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:55.818133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1069/2199] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:55 2022] PUT /identity/v3/projects/a24091b8fe594c3192007255f87f2fb4/users/a2aa2969716948be9aeab78be8220d45/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:55.828403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f7bd-d5c4-4022-bb31-6a9ae8583663 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:55.829630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f7bd-d5c4-4022-bb31-6a9ae8583663 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:55.830235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f7bd-d5c4-4022-bb31-6a9ae8583663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:55.877074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0317f7bd-d5c4-4022-bb31-6a9ae8583663 None None] MFA Rules not processed for user `a2aa2969716948be9aeab78be8220d45`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:55.878984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0317f7bd-d5c4-4022-bb31-6a9ae8583663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:55.942044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1131/2200] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:55.946621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.008460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cf62d-caf2-4d16-8a9b-186a94f1cbc0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.013121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.013770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.014053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.014328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.015019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.016952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.130117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1070/2201] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:55 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:56.134123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f7bd-d5c4-4022-bb31-6a9ae8583663 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.145712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f7bd-d5c4-4022-bb31-6a9ae8583663 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.201039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.201709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.202026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.202301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.202981 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.204970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.302989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1132/2202] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:56 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:56.307855 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.319694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78a23-7581-4d35-9858-a7e604f25093 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.324470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.325122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.325407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.325683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.326403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.328052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.338857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1071/2203] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:56.342741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.353235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dd63f-586c-4503-a2b4-dec1e22e234c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.357871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad4d0-14b2-402f-921a-2a49aeb8fcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.358554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad4d0-14b2-402f-921a-2a49aeb8fcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.358834 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad4d0-14b2-402f-921a-2a49aeb8fcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.359109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad4d0-14b2-402f-921a-2a49aeb8fcf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.359869 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ad4d0-14b2-402f-921a-2a49aeb8fcf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.361460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ad4d0-14b2-402f-921a-2a49aeb8fcf4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.372140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1133/2204] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:56.376101 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.386461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c78810-3092-4e13-b572-78e57b5d2063 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.391004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.391904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.392183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.392456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] PATH_INFO: `/v3/projects/de8e2f8671584525bac84a8d1cd8f947/users/764822f347d94a61bda2345e6138b5d0/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.407022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8e2f8671584525bac84a8d1cd8f947, user_id=764822f347d94a61bda2345e6138b5d0, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.409814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.510826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1072/2205] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:56 2022] PUT /identity/v3/projects/de8e2f8671584525bac84a8d1cd8f947/users/764822f347d94a61bda2345e6138b5d0/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:56.522033 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ddf75-5b6e-4743-8d40-83d48698f9d0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.522437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ddf75-5b6e-4743-8d40-83d48698f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.522716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ddf75-5b6e-4743-8d40-83d48698f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.548499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-113ddf75-5b6e-4743-8d40-83d48698f9d0 None None] MFA Rules not processed for user `764822f347d94a61bda2345e6138b5d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:56.549707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-113ddf75-5b6e-4743-8d40-83d48698f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:56.611565 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1134/2206] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:56.616510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.674587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc2730-911c-4c32-97a0-8b2d942f4335 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.680044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.680689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.681140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.681509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.682022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.683875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.763216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1073/2207] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:16:56 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:16:56.767210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ddf75-5b6e-4743-8d40-83d48698f9d0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.778173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ddf75-5b6e-4743-8d40-83d48698f9d0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.834745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.835412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.835723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.835998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.836688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.838623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.922160 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1135/2208] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:16:56 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:16:56.926769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.937161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a96a6e-3ecf-49f0-b259-9508980341be admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.942115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.942798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.943386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.943691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.944421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.946013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:56.959662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1074/2209] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:16:56.962506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:56.973695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf76f3-ddc1-4d49-b046-bc4530ba9884 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:56.978438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:56.979352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:56.979655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:56.979943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] PATH_INFO: `/v3/projects/055ac49c6f3541c0a3b29bdb330ca363/users/3842d8c6ded84bbaab7b944dc74c0890/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:56.995097 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055ac49c6f3541c0a3b29bdb330ca363, user_id=3842d8c6ded84bbaab7b944dc74c0890, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:56.997853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:57.070772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1136/2210] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:56 2022] PUT /identity/v3/projects/055ac49c6f3541c0a3b29bdb330ca363/users/3842d8c6ded84bbaab7b944dc74c0890/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:57.074758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:57.085154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d7e86-0d48-43ff-9403-89c9388a3a7b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:57.139912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa814b0-6e36-47ce-a4c9-f527d004c998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:57.140496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa814b0-6e36-47ce-a4c9-f527d004c998 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:57.140782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa814b0-6e36-47ce-a4c9-f527d004c998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:57.141056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa814b0-6e36-47ce-a4c9-f527d004c998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:57.141769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa814b0-6e36-47ce-a4c9-f527d004c998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:57.143370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa814b0-6e36-47ce-a4c9-f527d004c998 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:57.153301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1075/2211] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:16:57.157330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:57.167594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74bfc-1ec2-46e7-94f8-4db224999fa7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:57.172337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:57.173204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:57.173479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:57.173749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] PATH_INFO: `/v3/projects/055ac49c6f3541c0a3b29bdb330ca363/users/3842d8c6ded84bbaab7b944dc74c0890/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:57.176415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=055ac49c6f3541c0a3b29bdb330ca363, user_id=3842d8c6ded84bbaab7b944dc74c0890, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:57.179145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:57.189002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1137/2212] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:16:57 2022] PUT /identity/v3/projects/055ac49c6f3541c0a3b29bdb330ca363/users/3842d8c6ded84bbaab7b944dc74c0890/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:16:57.199768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de25b1b-80f1-4a7d-9246-582a12650973 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:57.200059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de25b1b-80f1-4a7d-9246-582a12650973 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:57.200339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de25b1b-80f1-4a7d-9246-582a12650973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:57.226377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6de25b1b-80f1-4a7d-9246-582a12650973 None None] MFA Rules not processed for user `3842d8c6ded84bbaab7b944dc74c0890`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:16:57.227556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6de25b1b-80f1-4a7d-9246-582a12650973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:57.293151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1076/2213] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:16:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:16:57.305106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:57.306792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:16:57.360496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db7dd4-bd2f-43df-84f8-c456e2c983ee admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:57.364853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:57.365489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:57.365773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:57.366047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:57.420042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:57.421412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:57.430726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1138/2214] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:16:59.401604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de25b1b-80f1-4a7d-9246-582a12650973 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:16:59.411772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de25b1b-80f1-4a7d-9246-582a12650973 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:16:59.417549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:16:59.417549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:16:59.417549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:16:59.417549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:16:59.472547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:16:59.474559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:16:59.483548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1077/2215] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:16:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:00.984187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:00.985588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:00.997422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550e60d-1385-4688-9f1c-bbe8a7731d65 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:01.001039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce25425-2d7b-4b93-bcfe-1c74a87506d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:01.001675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce25425-2d7b-4b93-bcfe-1c74a87506d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:01.001957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce25425-2d7b-4b93-bcfe-1c74a87506d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:01.002234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce25425-2d7b-4b93-bcfe-1c74a87506d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:01.011679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce25425-2d7b-4b93-bcfe-1c74a87506d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:01.013118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce25425-2d7b-4b93-bcfe-1c74a87506d0 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:01.023410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1139/2216] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:02.313978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:02.315572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:02.407750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d52c45-1743-4a15-989f-42656c16fe42 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:02.414277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:02.416114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:02.416114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:02.416352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] PATH_INFO: `/v3/users/7a3a047e0a1342ce91a8889f6b6c0b1e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:02.417720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3a047e0a1342ce91a8889f6b6c0b1e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:02.420262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:02.619650 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3a047e0a1342ce91a8889f6b6c0b1e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:02.687594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3a047e0a1342ce91a8889f6b6c0b1e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:02.689053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3a047e0a1342ce91a8889f6b6c0b1e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:02.702678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3a047e0a1342ce91a8889f6b6c0b1e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:02.710524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3a047e0a1342ce91a8889f6b6c0b1e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:02.716959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1078/2217] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:17:02 2022] DELETE /identity/v3/users/7a3a047e0a1342ce91a8889f6b6c0b1e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:02.721519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:02.722047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:02.722205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:02.740334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:02.741673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:02.803363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1140/2218] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:02.813609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:02.815017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:02.888586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10ea0c-b767-40a4-b24e-6ad5ff5ff419 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:02.892434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fef97b-f77e-4018-8ea0-024b54d251b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:02.893098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fef97b-f77e-4018-8ea0-024b54d251b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:02.893357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fef97b-f77e-4018-8ea0-024b54d251b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:02.893627 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fef97b-f77e-4018-8ea0-024b54d251b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:02.904521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fef97b-f77e-4018-8ea0-024b54d251b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:02.905958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fef97b-f77e-4018-8ea0-024b54d251b1 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:02.916373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1079/2219] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:17:03.465059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:03.466850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:03.479711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddda5ef-1009-45b2-a177-9b0442c13e69 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:03.484673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:03.485776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:03.486240 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:03.486710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] PATH_INFO: `/v3/projects/147da5d5de00409eb9b982affecf976b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:03.488134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147da5d5de00409eb9b982affecf976b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:03.492321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:03.600819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147da5d5de00409eb9b982affecf976b', 'request_id': 'req-a08b8c05-0016-42b5-848e-777654408377'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:03.606365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147da5d5de00409eb9b982affecf976b', 'request_id': 'req-a08b8c05-0016-42b5-848e-777654408377'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:03.612734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147da5d5de00409eb9b982affecf976b', 'request_id': 'req-a08b8c05-0016-42b5-848e-777654408377'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:03.696662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] The token cache is being invalidate because project 147da5d5de00409eb9b982affecf976b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:17:03.696951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:03.697581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a08b8c05-0016-42b5-848e-777654408377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:03.699151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1141/2220] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:17:03 2022] DELETE /identity/v3/projects/147da5d5de00409eb9b982affecf976b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:17:03.704166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c55154-f954-45d9-ac13-8991a96950be None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:03.704626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c55154-f954-45d9-ac13-8991a96950be None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:03.704755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c55154-f954-45d9-ac13-8991a96950be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:03.724452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f7c55154-f954-45d9-ac13-8991a96950be None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:03.725902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7c55154-f954-45d9-ac13-8991a96950be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:03.790801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1080/2221] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:03.798164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481d86-b612-4cee-8aed-bd378ec08bcd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:03.798633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481d86-b612-4cee-8aed-bd378ec08bcd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:03.803542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481d86-b612-4cee-8aed-bd378ec08bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:03.818542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5a481d86-b612-4cee-8aed-bd378ec08bcd None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:03.819835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a481d86-b612-4cee-8aed-bd378ec08bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:03.827717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1142/2222] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:03.831856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c55154-f954-45d9-ac13-8991a96950be None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:03.845035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c55154-f954-45d9-ac13-8991a96950be None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:03.846750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:03.847827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:03.847827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:03.848007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:03.848725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:17:03.849032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:03.850888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:03.863156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1081/2223] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:17:03.868087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a481d86-b612-4cee-8aed-bd378ec08bcd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:03.877697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a481d86-b612-4cee-8aed-bd378ec08bcd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:03.883156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:03.883840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:03.884122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:03.885123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:03.885123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:03.886954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:03.965724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1143/2224] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:17:03 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:17:03.969802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:03.980384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769570f-1dde-4969-9d3c-47ae9c690b07 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:04.037468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:04.038134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:04.038413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:04.038712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:04.039398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:04.041394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:04.159516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1082/2225] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:17:03 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:17:04.164059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:04.174004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942b956-e317-40a9-aa05-b3fcf8c38b97 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:04.178555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:04.179197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:04.179476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:04.179793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:04.180511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:04.182080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:04.191997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1144/2226] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:17:04.196912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:04.207998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c4a6c-89ed-4943-ad4d-c57680a83e60 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:04.216388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6838239-e875-46a8-aebe-d2514e84babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:04.216388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6838239-e875-46a8-aebe-d2514e84babd admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:04.216388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6838239-e875-46a8-aebe-d2514e84babd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:04.216388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6838239-e875-46a8-aebe-d2514e84babd admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:04.216871 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6838239-e875-46a8-aebe-d2514e84babd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:04.217077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6838239-e875-46a8-aebe-d2514e84babd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:04.227528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1083/2227] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:17:04.231700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:04.242231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d846-8c0c-4e6c-8a24-ebc5ff8b8b78 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:04.252712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:04.253620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:04.253903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:04.254176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] PATH_INFO: `/v3/projects/6b9dc9e1c86448bda7d81301dd230a9f/users/d35918b7065e4cf98c309c933055d96f/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:04.275484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b9dc9e1c86448bda7d81301dd230a9f, user_id=d35918b7065e4cf98c309c933055d96f, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:04.278265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:04.344987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1145/2228] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:17:04 2022] PUT /identity/v3/projects/6b9dc9e1c86448bda7d81301dd230a9f/users/d35918b7065e4cf98c309c933055d96f/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:04.355481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:04.355813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:04.356092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:04.385059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] MFA Rules not processed for user `d35918b7065e4cf98c309c933055d96f`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:04.386279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:04.453118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1084/2229] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:04.500968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:04.502817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:04.574616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcda07-ad46-4ebe-8313-39e65ab1e44f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:04.578776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6a8f6-dba9-4c4d-abca-d0d3bbaaecf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:04.579433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6a8f6-dba9-4c4d-abca-d0d3bbaaecf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:04.579845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6a8f6-dba9-4c4d-abca-d0d3bbaaecf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:04.580122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6a8f6-dba9-4c4d-abca-d0d3bbaaecf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:04.589456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6a8f6-dba9-4c4d-abca-d0d3bbaaecf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:04.590937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6a8f6-dba9-4c4d-abca-d0d3bbaaecf4 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:04.600251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1146/2230] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:05.816277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:05.817721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:05.886511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757bdb7-f254-4479-ae65-ded52a530580 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:05.893005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:05.893874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:05.894151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:05.894418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] PATH_INFO: `/v3/users/a2aa2969716948be9aeab78be8220d45` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:05.895585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2aa2969716948be9aeab78be8220d45)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:05.898542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:06.078410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2aa2969716948be9aeab78be8220d45'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:06.150267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2aa2969716948be9aeab78be8220d45'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:06.150892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2aa2969716948be9aeab78be8220d45'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:06.162818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2aa2969716948be9aeab78be8220d45'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:06.169499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2aa2969716948be9aeab78be8220d45'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:06.175265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1085/2231] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:17:05 2022] DELETE /identity/v3/users/a2aa2969716948be9aeab78be8220d45 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:06.180542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581da10-1786-47bb-83d5-fb911275386f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:06.180992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581da10-1786-47bb-83d5-fb911275386f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:06.181200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581da10-1786-47bb-83d5-fb911275386f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:06.198493 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a581da10-1786-47bb-83d5-fb911275386f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:06.199737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a581da10-1786-47bb-83d5-fb911275386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:06.260630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1147/2232] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:06.270407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:06.272761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:06.332961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5593d5e-1920-4005-9bbd-1d291c8fc5db admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:06.337815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:06.338456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:06.339112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:06.339112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:06.348744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:06.350485 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:06.361108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1086/2233] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:06.892070 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581da10-1786-47bb-83d5-fb911275386f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:06.892891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a581da10-1786-47bb-83d5-fb911275386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:06.917294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581da10-1786-47bb-83d5-fb911275386f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:06.936599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:06.937479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:06.937763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:06.938040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] PATH_INFO: `/v3/projects/a24091b8fe594c3192007255f87f2fb4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:06.940732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24091b8fe594c3192007255f87f2fb4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:06.942835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:07.073546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24091b8fe594c3192007255f87f2fb4', 'request_id': 'req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.084869 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24091b8fe594c3192007255f87f2fb4', 'request_id': 'req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.107857 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24091b8fe594c3192007255f87f2fb4', 'request_id': 'req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.169973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] The token cache is being invalidate because project a24091b8fe594c3192007255f87f2fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:17:07.170084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.170712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.172388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1148/2234] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:17:06 2022] DELETE /identity/v3/projects/a24091b8fe594c3192007255f87f2fb4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:07.176492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:07.187397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f45be7-1645-4c10-9bc6-1614a9fa3d3c service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:07.244997 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:07.245868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:07.246148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:07.246419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] PATH_INFO: `/v3/users/764822f347d94a61bda2345e6138b5d0` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:07.247646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=764822f347d94a61bda2345e6138b5d0)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:07.249630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:07.418692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764822f347d94a61bda2345e6138b5d0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.493093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764822f347d94a61bda2345e6138b5d0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.493695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764822f347d94a61bda2345e6138b5d0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.505389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764822f347d94a61bda2345e6138b5d0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.511910 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764822f347d94a61bda2345e6138b5d0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:07.517780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1087/2235] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:17:07 2022] DELETE /identity/v3/users/764822f347d94a61bda2345e6138b5d0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:07.926180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:07.926180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:07.939166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc702ad-2dae-4b42-bfc6-90fd6ae34386 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:07.995915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:07.996767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:07.997043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:07.997314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] PATH_INFO: `/v3/projects/de8e2f8671584525bac84a8d1cd8f947` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:07.998457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8e2f8671584525bac84a8d1cd8f947)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:08.000395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:08.118332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8e2f8671584525bac84a8d1cd8f947', 'request_id': 'req-d932f335-72c7-473c-9c3a-1a7557f5ac22'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.123563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8e2f8671584525bac84a8d1cd8f947', 'request_id': 'req-d932f335-72c7-473c-9c3a-1a7557f5ac22'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.129824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8e2f8671584525bac84a8d1cd8f947', 'request_id': 'req-d932f335-72c7-473c-9c3a-1a7557f5ac22'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.216201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] The token cache is being invalidate because project de8e2f8671584525bac84a8d1cd8f947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:17:08.216490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.217107 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.218701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1149/2236] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:17:07 2022] DELETE /identity/v3/projects/de8e2f8671584525bac84a8d1cd8f947 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:08.222644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:08.224117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:08.237218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2706c2a-dba5-4ad5-9443-fb73729d1f7a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:08.293122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:08.293998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:08.294276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:08.294547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] PATH_INFO: `/v3/users/3842d8c6ded84bbaab7b944dc74c0890` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:08.295746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3842d8c6ded84bbaab7b944dc74c0890)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:08.297751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:08.474205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3842d8c6ded84bbaab7b944dc74c0890'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.553949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3842d8c6ded84bbaab7b944dc74c0890'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.554586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3842d8c6ded84bbaab7b944dc74c0890'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.566069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3842d8c6ded84bbaab7b944dc74c0890'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.573367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1a4d12b4-1d0b-4a4d-9371-6674178f00fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3842d8c6ded84bbaab7b944dc74c0890'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:08.579266 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1088/2237] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:17:08 2022] DELETE /identity/v3/users/3842d8c6ded84bbaab7b944dc74c0890 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:08.960770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:08.962239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:08.975832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932f335-72c7-473c-9c3a-1a7557f5ac22 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:09.034236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:09.035123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.035398 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.035698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] PATH_INFO: `/v3/projects/055ac49c6f3541c0a3b29bdb330ca363` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.036905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055ac49c6f3541c0a3b29bdb330ca363)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:09.039027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:09.144605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055ac49c6f3541c0a3b29bdb330ca363', 'request_id': 'req-9c512d52-e8dc-45cd-aee8-0a7606cecac8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:09.150499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055ac49c6f3541c0a3b29bdb330ca363', 'request_id': 'req-9c512d52-e8dc-45cd-aee8-0a7606cecac8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:09.156714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055ac49c6f3541c0a3b29bdb330ca363', 'request_id': 'req-9c512d52-e8dc-45cd-aee8-0a7606cecac8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:09.217437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] The token cache is being invalidate because project 055ac49c6f3541c0a3b29bdb330ca363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:17:09.217722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:09.218330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9c512d52-e8dc-45cd-aee8-0a7606cecac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:09.219894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1150/2238] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:17:08 2022] DELETE /identity/v3/projects/055ac49c6f3541c0a3b29bdb330ca363 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:09.225529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e2d44-7988-428e-8c6f-748fd3e06c9a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.226161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e2d44-7988-428e-8c6f-748fd3e06c9a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.226161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e2d44-7988-428e-8c6f-748fd3e06c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.244496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7b6e2d44-7988-428e-8c6f-748fd3e06c9a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:09.245609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7b6e2d44-7988-428e-8c6f-748fd3e06c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:09.301088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1089/2239] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:09.309346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1581bd2-3d07-4583-92f4-0847c308378d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.309676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1581bd2-3d07-4583-92f4-0847c308378d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.309959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1581bd2-3d07-4583-92f4-0847c308378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.329238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e1581bd2-3d07-4583-92f4-0847c308378d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:09.330442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e1581bd2-3d07-4583-92f4-0847c308378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:09.339038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1151/2240] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:09.343295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e2d44-7988-428e-8c6f-748fd3e06c9a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:09.352880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e2d44-7988-428e-8c6f-748fd3e06c9a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:09.357536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:09.358189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.358466 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.358735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.359487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:17:09.359816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:09.361395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:09.377032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1090/2241] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:17:09.378118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1581bd2-3d07-4583-92f4-0847c308378d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:09.388146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1581bd2-3d07-4583-92f4-0847c308378d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:09.393009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:09.393668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.393952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.394227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.394929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:09.397741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:09.507014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1152/2242] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:17:09 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:17:09.510744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:09.520370 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd379d7f-8789-4d2b-a7c7-1c34637f5942 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:09.574283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:09.574939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.575217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.575489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.576206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:09.578143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:09.667449 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1091/2243] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:17:09 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:17:09.671594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:09.682282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925672d3-df87-49df-a60c-1261ee2c9664 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:09.686915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:09.687567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.687882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.688161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.688885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:09.690709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:09.703018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1153/2244] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:17:09.707234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:09.717120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18df3d-31f0-4ab1-bb0e-90d93eba11ed admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:09.721937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc2682-934b-45de-9ae7-0d242aa5fb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:09.722710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc2682-934b-45de-9ae7-0d242aa5fb3b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.723349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc2682-934b-45de-9ae7-0d242aa5fb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.723542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc2682-934b-45de-9ae7-0d242aa5fb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.724374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc2682-934b-45de-9ae7-0d242aa5fb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:09.726209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc2682-934b-45de-9ae7-0d242aa5fb3b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:09.736729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1092/2245] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:17:09.741380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:09.751036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e6daa-5b4d-4ee1-affb-43c4e606bc04 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:09.755804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:09.757276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.757501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.757785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] PATH_INFO: `/v3/projects/68e10cc41f9f48a2b32da2f5cc565318/users/476846e5ab6a45539ccaf9ad0cf67418/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.773216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e10cc41f9f48a2b32da2f5cc565318, user_id=476846e5ab6a45539ccaf9ad0cf67418, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:09.775967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:09.845526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1154/2246] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:17:09 2022] PUT /identity/v3/projects/68e10cc41f9f48a2b32da2f5cc565318/users/476846e5ab6a45539ccaf9ad0cf67418/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:09.857298 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8d40d-9f3c-4d5c-acc0-3974dd6fc7bc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:09.857471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8d40d-9f3c-4d5c-acc0-3974dd6fc7bc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:09.857749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8d40d-9f3c-4d5c-acc0-3974dd6fc7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:09.884356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-39a8d40d-9f3c-4d5c-acc0-3974dd6fc7bc None None] MFA Rules not processed for user `476846e5ab6a45539ccaf9ad0cf67418`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:09.885576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-39a8d40d-9f3c-4d5c-acc0-3974dd6fc7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:09.946076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1093/2247] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:09.957019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:09.958711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:10.014245 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08664af4-a02d-4848-863a-409e3ad75494 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:10.018118 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0677f-27cd-4626-995e-604448dab953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:10.018771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0677f-27cd-4626-995e-604448dab953 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:10.019058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0677f-27cd-4626-995e-604448dab953 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:10.019369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0677f-27cd-4626-995e-604448dab953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:10.029040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0677f-27cd-4626-995e-604448dab953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:10.030472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0677f-27cd-4626-995e-604448dab953 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:10.039621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1155/2248] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:13.202725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51481fab-3162-4ccb-aedc-1d7488105a81 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:13.203254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51481fab-3162-4ccb-aedc-1d7488105a81 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:13.203346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51481fab-3162-4ccb-aedc-1d7488105a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:13.221317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-51481fab-3162-4ccb-aedc-1d7488105a81 None None] MFA Rules not processed for user `97021ddaffff4e088961ba1873a8e697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:13.226118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51481fab-3162-4ccb-aedc-1d7488105a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:13.274492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1094/2249] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:14.355285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71787750-68a4-42c1-925a-0cabf08032cd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:14.355692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71787750-68a4-42c1-925a-0cabf08032cd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:14.355888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71787750-68a4-42c1-925a-0cabf08032cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:14.376966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-71787750-68a4-42c1-925a-0cabf08032cd None None] MFA Rules not processed for user `d38cdf40e33b47d8aa79e029cded7503`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:14.378385 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-71787750-68a4-42c1-925a-0cabf08032cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:14.438859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1156/2250] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:15.197623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51481fab-3162-4ccb-aedc-1d7488105a81 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:15.263745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51481fab-3162-4ccb-aedc-1d7488105a81 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:15.268773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:15.269782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:15.270122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:15.270558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] PATH_INFO: `/v3/users/476846e5ab6a45539ccaf9ad0cf67418` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:15.272557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] RBAC: Authorizing `identity:delete_user(user_id=476846e5ab6a45539ccaf9ad0cf67418)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:15.275509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:15.439325 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '476846e5ab6a45539ccaf9ad0cf67418'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:15.536887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '476846e5ab6a45539ccaf9ad0cf67418'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:15.540092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '476846e5ab6a45539ccaf9ad0cf67418'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:15.554012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '476846e5ab6a45539ccaf9ad0cf67418'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:15.560892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '476846e5ab6a45539ccaf9ad0cf67418'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:15.567335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1095/2251] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:17:15 2022] DELETE /identity/v3/users/476846e5ab6a45539ccaf9ad0cf67418 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:15.572187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:15.573127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:15.573623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:15.591964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:15.593243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:15.657114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1157/2252] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:15.667611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:15.669086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:15.723965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c737-e273-4ff7-9a49-1fd8c405717b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:15.728440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:15.729141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:15.729419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:15.729694 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:15.738845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:15.740367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:15.749661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1096/2253] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:17:16.234953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:16.236464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:16.251879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9432-5f25-4091-a466-a935812a07b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:16.257143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:16.258297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:16.258755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:16.259185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] PATH_INFO: `/v3/projects/68e10cc41f9f48a2b32da2f5cc565318` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:16.260770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e10cc41f9f48a2b32da2f5cc565318)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:16.262992 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:16.418151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e10cc41f9f48a2b32da2f5cc565318', 'request_id': 'req-eaf8a2cd-9f82-4366-b375-3aea8e760951'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:16.424976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e10cc41f9f48a2b32da2f5cc565318', 'request_id': 'req-eaf8a2cd-9f82-4366-b375-3aea8e760951'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:16.432494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e10cc41f9f48a2b32da2f5cc565318', 'request_id': 'req-eaf8a2cd-9f82-4366-b375-3aea8e760951'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:16.553331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] The token cache is being invalidate because project 68e10cc41f9f48a2b32da2f5cc565318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:17:16.553607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:16.554225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eaf8a2cd-9f82-4366-b375-3aea8e760951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:16.555925 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1158/2254] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:17:16 2022] DELETE /identity/v3/projects/68e10cc41f9f48a2b32da2f5cc565318 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:28.791307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:28.792732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:28.805944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4429c-db68-4528-9d43-9d038955879b service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:28.842650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:28.843154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:28.843644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:28.874138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:28.875019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:28.875295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:28.875562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] PATH_INFO: `/v3/users/d35918b7065e4cf98c309c933055d96f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:28.876791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35918b7065e4cf98c309c933055d96f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:28.876918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:28.878125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:28.878740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:28.946619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1159/2255] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:17:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:28.996155 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:28.997764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:29.075058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343cc7b-35bb-47e3-bb3d-c2870ebfad56 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:29.080090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96bc7e-2864-46bc-b4b7-c5b2c3f13cde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:29.080744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bc7e-2864-46bc-b4b7-c5b2c3f13cde service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:29.081031 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bc7e-2864-46bc-b4b7-c5b2c3f13cde service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:29.081303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bc7e-2864-46bc-b4b7-c5b2c3f13cde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:29.089872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96bc7e-2864-46bc-b4b7-c5b2c3f13cde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:29.091462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96bc7e-2864-46bc-b4b7-c5b2c3f13cde service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:29.099295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1160/2256] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:17:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 16 15:17:29.121700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35918b7065e4cf98c309c933055d96f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:29.213824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35918b7065e4cf98c309c933055d96f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:29.214515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35918b7065e4cf98c309c933055d96f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:29.228615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35918b7065e4cf98c309c933055d96f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:29.235809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35918b7065e4cf98c309c933055d96f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:29.241858 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1097/2257] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:17:28 2022] DELETE /identity/v3/users/d35918b7065e4cf98c309c933055d96f => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:29.246588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:29.246860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:29.247145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:29.266537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:29.267787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:29.335459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1161/2258] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:29.343500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:29.345057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:29.424356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb180357-fc5f-4a54-b13c-f4819bd32aec admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:29.429878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c1c4a-200e-4d62-83f5-7634a5647e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:29.432225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c1c4a-200e-4d62-83f5-7634a5647e08 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:29.432516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c1c4a-200e-4d62-83f5-7634a5647e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:29.432797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c1c4a-200e-4d62-83f5-7634a5647e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:29.444803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c1c4a-200e-4d62-83f5-7634a5647e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:29.447515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c1c4a-200e-4d62-83f5-7634a5647e08 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:29.471246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1098/2259] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:30.009772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:30.011250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:30.042479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4b9c-0138-439d-b0dc-64ed00c1ec1c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:30.051039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:30.051311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:30.052027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:30.052484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] PATH_INFO: `/v3/projects/6b9dc9e1c86448bda7d81301dd230a9f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:30.053989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b9dc9e1c86448bda7d81301dd230a9f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:30.056682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:30.255010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b9dc9e1c86448bda7d81301dd230a9f', 'request_id': 'req-f3743d76-0eff-4a2f-ade0-633957968c97'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:30.266146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b9dc9e1c86448bda7d81301dd230a9f', 'request_id': 'req-f3743d76-0eff-4a2f-ade0-633957968c97'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:30.280809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b9dc9e1c86448bda7d81301dd230a9f', 'request_id': 'req-f3743d76-0eff-4a2f-ade0-633957968c97'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:30.382747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] The token cache is being invalidate because project 6b9dc9e1c86448bda7d81301dd230a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:17:30.383444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:30.385154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f3743d76-0eff-4a2f-ade0-633957968c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:17:30.387284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1162/2260] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:17:30 2022] DELETE /identity/v3/projects/6b9dc9e1c86448bda7d81301dd230a9f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:17:46.449237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149a8f-f5db-4914-80ad-8cc276d90aae None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:46.449838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149a8f-f5db-4914-80ad-8cc276d90aae None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:46.449838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149a8f-f5db-4914-80ad-8cc276d90aae None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:46.451162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1099/2261] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:17:46.456964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5309-39b2-47c3-88fe-746ed215769e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:46.457451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5309-39b2-47c3-88fe-746ed215769e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:46.457546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5309-39b2-47c3-88fe-746ed215769e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:46.508951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ed0d5309-39b2-47c3-88fe-746ed215769e None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:46.510188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ed0d5309-39b2-47c3-88fe-746ed215769e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:46.585019 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1163/2262] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:17:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:46.597798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67149a8f-f5db-4914-80ad-8cc276d90aae None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:46.599475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-67149a8f-f5db-4914-80ad-8cc276d90aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:46.678004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67149a8f-f5db-4914-80ad-8cc276d90aae None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:46.682130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76397599-8647-4d2c-b4aa-642164431831 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:46.682870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76397599-8647-4d2c-b4aa-642164431831 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:46.683143 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76397599-8647-4d2c-b4aa-642164431831 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:46.683413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76397599-8647-4d2c-b4aa-642164431831 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:46.693452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76397599-8647-4d2c-b4aa-642164431831 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:46.694929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76397599-8647-4d2c-b4aa-642164431831 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:46.706006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1100/2263] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:46.920665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eb22c-8830-4c64-9418-cbfb34e78a19 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:46.921002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eb22c-8830-4c64-9418-cbfb34e78a19 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:46.921285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eb22c-8830-4c64-9418-cbfb34e78a19 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:46.922688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1164/2264] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:17:46.927359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003aa5-1b95-4144-88b1-dd92d79aa489 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:46.927669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003aa5-1b95-4144-88b1-dd92d79aa489 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:46.927962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003aa5-1b95-4144-88b1-dd92d79aa489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:46.947004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-87003aa5-1b95-4144-88b1-dd92d79aa489 None None] MFA Rules not processed for user `76b2a2d381a14f5fb04e721905b36ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:46.948317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-87003aa5-1b95-4144-88b1-dd92d79aa489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:46.996785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1101/2265] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:17:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:47.009060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900eb22c-8830-4c64-9418-cbfb34e78a19 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:47.010962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-900eb22c-8830-4c64-9418-cbfb34e78a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:47.024138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900eb22c-8830-4c64-9418-cbfb34e78a19 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:47.028464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64388d-a2cf-4405-9778-94260e764faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:47.029094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64388d-a2cf-4405-9778-94260e764faa service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:47.029370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64388d-a2cf-4405-9778-94260e764faa service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:47.029639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64388d-a2cf-4405-9778-94260e764faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:47.036571 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64388d-a2cf-4405-9778-94260e764faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:47.037990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64388d-a2cf-4405-9778-94260e764faa service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:47.045562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1165/2266] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:17:47.129181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ec2f0-b4a7-4d0c-a65a-ae1283bdfeea None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:47.129474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ec2f0-b4a7-4d0c-a65a-ae1283bdfeea None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:47.129752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ec2f0-b4a7-4d0c-a65a-ae1283bdfeea None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:47.131133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1102/2267] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:17:47.137036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df76b2-b8ec-42ad-8dbd-af301fec00d4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:47.137373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df76b2-b8ec-42ad-8dbd-af301fec00d4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:47.137648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df76b2-b8ec-42ad-8dbd-af301fec00d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:47.156985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-15df76b2-b8ec-42ad-8dbd-af301fec00d4 None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:47.158187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-15df76b2-b8ec-42ad-8dbd-af301fec00d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:47.168040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1166/2268] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:17:47.178352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ec2f0-b4a7-4d0c-a65a-ae1283bdfeea None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:47.189014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ec2f0-b4a7-4d0c-a65a-ae1283bdfeea None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:47.192593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:47.193765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:47.193765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:47.194136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:47.205303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:47.206727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:47.218366 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1103/2269] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:17:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:17:56.839046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b516bab-2ee0-42d3-8fe5-ff7d2faf60d9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:56.839500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b516bab-2ee0-42d3-8fe5-ff7d2faf60d9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:56.839760 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b516bab-2ee0-42d3-8fe5-ff7d2faf60d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:56.885972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4b516bab-2ee0-42d3-8fe5-ff7d2faf60d9 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:17:56.887134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4b516bab-2ee0-42d3-8fe5-ff7d2faf60d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:56.948865 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1167/2270] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:17:56.960823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:17:56.962311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:17:57.051746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40729a2d-d712-4480-9493-63bf655d30e3 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:17:57.056669 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:17:57.057314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:17:57.057792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:17:57.058660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:17:57.067089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:17:57.068733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:17:57.077068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1104/2271] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:17:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:18:18.089032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:18:18.090146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:18:18.090379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:18:18.108581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:18:18.109763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:18:18.117023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1168/2272] 10.222.0.71 () {58 vars in 967 bytes} [Mon May 16 15:18:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:18:18.130083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:18:18.140847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674693a-21c8-4607-b6e8-f7d49c6c11e2 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:18:18.145698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cab260-f351-4f55-b7f0-bb44d54ce817 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:18:18.146577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cab260-f351-4f55-b7f0-bb44d54ce817 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:18:18.146854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cab260-f351-4f55-b7f0-bb44d54ce817 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:18:18.147141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cab260-f351-4f55-b7f0-bb44d54ce817 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:18:18.154833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cab260-f351-4f55-b7f0-bb44d54ce817 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:18:18.156420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cab260-f351-4f55-b7f0-bb44d54ce817 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:18:18.164103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1105/2273] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:18:24.793348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:18:24.794818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:18:24.890811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957ad9b-85f7-4919-b5a1-732a5f431e8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:18:24.896544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0513fe-9234-4ad6-b898-b904092f2427 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:18:24.897523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0513fe-9234-4ad6-b898-b904092f2427 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:18:24.897897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0513fe-9234-4ad6-b898-b904092f2427 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:18:24.899535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0513fe-9234-4ad6-b898-b904092f2427 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:18:24.901179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c0513fe-9234-4ad6-b898-b904092f2427 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:18:24.913638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0513fe-9234-4ad6-b898-b904092f2427 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:18:24.915285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0513fe-9234-4ad6-b898-b904092f2427 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:18:24.926241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1169/2274] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:18:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:18:41.100346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf54d0-d838-4a2d-9f73-1dc29c3d5ccf None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:18:41.103258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf54d0-d838-4a2d-9f73-1dc29c3d5ccf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:18:41.103258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf54d0-d838-4a2d-9f73-1dc29c3d5ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:18:41.119281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4dcf54d0-d838-4a2d-9f73-1dc29c3d5ccf None None] MFA Rules not processed for user `d38cdf40e33b47d8aa79e029cded7503`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:18:41.120504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4dcf54d0-d838-4a2d-9f73-1dc29c3d5ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:18:41.175279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1106/2275] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:18:51.764094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c22af-7fa3-4731-a35e-adda626d84a4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:18:51.765115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c22af-7fa3-4731-a35e-adda626d84a4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:18:51.765115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c22af-7fa3-4731-a35e-adda626d84a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:18:51.784064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0c6c22af-7fa3-4731-a35e-adda626d84a4 None None] MFA Rules not processed for user `97021ddaffff4e088961ba1873a8e697`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:18:51.786049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0c6c22af-7fa3-4731-a35e-adda626d84a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:18:51.846659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1170/2276] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:19:11.774862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0b2d-edf2-4deb-8511-0bd82d7a0dfe None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:19:11.775390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0b2d-edf2-4deb-8511-0bd82d7a0dfe None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:19:11.775390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0b2d-edf2-4deb-8511-0bd82d7a0dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:19:11.792473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-051a0b2d-edf2-4deb-8511-0bd82d7a0dfe None None] MFA Rules not processed for user `7b559cc982a846bea7f7f2077b7c455b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:19:11.793649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-051a0b2d-edf2-4deb-8511-0bd82d7a0dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:19:11.845699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1107/2277] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:19:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:19:54.648757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f771ce-f6c0-4be1-a045-0f60d374254d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:19:54.649386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f771ce-f6c0-4be1-a045-0f60d374254d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:19:54.649386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f771ce-f6c0-4be1-a045-0f60d374254d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:19:54.672512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-67f771ce-f6c0-4be1-a045-0f60d374254d None None] MFA Rules not processed for user `426aa73007cd4ca3b8c165b87f69c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:19:54.673952 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-67f771ce-f6c0-4be1-a045-0f60d374254d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:19:54.748907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1171/2278] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:19:59.228222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fac400-7b6f-4a48-991a-2024132152cc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:19:59.228709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fac400-7b6f-4a48-991a-2024132152cc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:19:59.228806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fac400-7b6f-4a48-991a-2024132152cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:19:59.246341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-63fac400-7b6f-4a48-991a-2024132152cc None None] MFA Rules not processed for user `d38cdf40e33b47d8aa79e029cded7503`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:19:59.247546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-63fac400-7b6f-4a48-991a-2024132152cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:19:59.254549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1108/2279] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:20:10.378954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f771ce-f6c0-4be1-a045-0f60d374254d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:10.379822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fac400-7b6f-4a48-991a-2024132152cc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:10.389245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fac400-7b6f-4a48-991a-2024132152cc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:10.389433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f771ce-f6c0-4be1-a045-0f60d374254d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:10.393202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:10.393541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:10.394216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:10.394359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:10.394661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:10.394935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:10.395063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:10.395695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:10.397144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:10.398219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:10.412631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:10.413015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:10.414218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:10.415017 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:10.427459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1109/2280] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:20:10.427734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1172/2281] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:14.166214 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:14.168162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:14.178505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087adefa-dfc0-4271-9a8a-d5e98801fc43 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:14.182697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:14.183344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:14.183668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:14.183955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:14.185890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:14.250369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:14.251878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:14.263469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1110/2282] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:18.602234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:18.604387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:18.621098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e9b11-2116-4126-8af6-b8752c19a38d service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:18.626697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:18.626697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:18.626697 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:18.626980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] PATH_INFO: `/v3/users/97021ddaffff4e088961ba1873a8e697` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:18.628119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97021ddaffff4e088961ba1873a8e697)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:18.630022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:18.791481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97021ddaffff4e088961ba1873a8e697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:18.863453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97021ddaffff4e088961ba1873a8e697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:18.871396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97021ddaffff4e088961ba1873a8e697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:18.872116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97021ddaffff4e088961ba1873a8e697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:18.887356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d5ebdab5-ae32-4602-a98d-fc30f0f316b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97021ddaffff4e088961ba1873a8e697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:18.893899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1173/2283] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:20:18 2022] DELETE /identity/v3/users/97021ddaffff4e088961ba1873a8e697 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:19.139616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:19.151776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f57ec3-3f0a-4d60-b28b-251c20901afd service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:19.215772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:19.216664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.216953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.217225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] PATH_INFO: `/v3/projects/e1e340fabe31456cb021fcd0b5f505b5` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.218441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e340fabe31456cb021fcd0b5f505b5)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:19.220386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:19.324812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e340fabe31456cb021fcd0b5f505b5', 'request_id': 'req-e4187199-90e3-44fa-a7be-be766200fbe6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:19.428786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e340fabe31456cb021fcd0b5f505b5', 'request_id': 'req-e4187199-90e3-44fa-a7be-be766200fbe6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:19.434451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e340fabe31456cb021fcd0b5f505b5', 'request_id': 'req-e4187199-90e3-44fa-a7be-be766200fbe6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:19.445321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] The token cache is being invalidate because project e1e340fabe31456cb021fcd0b5f505b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:20:19.445608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:19.446315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e4187199-90e3-44fa-a7be-be766200fbe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:19.448205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1111/2284] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:20:19 2022] DELETE /identity/v3/projects/e1e340fabe31456cb021fcd0b5f505b5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:19.453081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc01b1-e095-45dc-8f73-eaec81619d1f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.453365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc01b1-e095-45dc-8f73-eaec81619d1f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.453640 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc01b1-e095-45dc-8f73-eaec81619d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.471714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8efc01b1-e095-45dc-8f73-eaec81619d1f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:19.472892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8efc01b1-e095-45dc-8f73-eaec81619d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:19.529958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1174/2285] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:19.537708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcf502-5c44-44b2-84e0-bfc05f1abc36 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.538000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcf502-5c44-44b2-84e0-bfc05f1abc36 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.538277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcf502-5c44-44b2-84e0-bfc05f1abc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.556826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7cbcf502-5c44-44b2-84e0-bfc05f1abc36 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:19.558025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7cbcf502-5c44-44b2-84e0-bfc05f1abc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:19.565842 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1112/2286] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:19.569902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc01b1-e095-45dc-8f73-eaec81619d1f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:19.579563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc01b1-e095-45dc-8f73-eaec81619d1f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:19.584260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:19.584909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.585295 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.585576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.586328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:20:19.586629 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:19.589076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:19.600259 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1175/2287] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:20:19.605024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcf502-5c44-44b2-84e0-bfc05f1abc36 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:19.614284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcf502-5c44-44b2-84e0-bfc05f1abc36 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:19.619247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:19.619931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.620218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.620493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.621465 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:19.623365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:19.709905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1113/2288] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:20:19 2022] POST /identity/v3/projects => generated 357 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:20:19.714687 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:19.724732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a13528-8aa8-44c9-8256-3dad3939d90d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:19.780486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:19.781147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.781431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.781706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.782401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:19.784361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:19.851585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1176/2289] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:20:19 2022] POST /identity/v3/users => generated 279 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:20:19.855874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:19.865826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a4c9a-ab71-4fbe-ba76-73e5bddb1085 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:19.870390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:19.871043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.871322 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.871595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.872351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:19.873931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:19.884930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1114/2290] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:19.888931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:19.899283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef01e-f47f-44e1-b5eb-1675e4fcbc47 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:19.904808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa206c-fb59-4589-8261-291f78a160a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:19.905464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa206c-fb59-4589-8261-291f78a160a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.905775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa206c-fb59-4589-8261-291f78a160a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.906047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa206c-fb59-4589-8261-291f78a160a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.906781 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa206c-fb59-4589-8261-291f78a160a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:19.908369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa206c-fb59-4589-8261-291f78a160a5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:19.919551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1177/2291] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:19.923320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:19.935024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71847d-4191-4e92-a050-09038926a3f9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:19.939610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:19.940821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:19.941105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:19.941379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] PATH_INFO: `/v3/projects/b87bacc307f64fc0be785e3c1da6bcad/users/03f5d87a43674d67b43082c23c068d83/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:19.956769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87bacc307f64fc0be785e3c1da6bcad, user_id=03f5d87a43674d67b43082c23c068d83, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:19.959426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.040646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1115/2292] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:20:19 2022] PUT /identity/v3/projects/b87bacc307f64fc0be785e3c1da6bcad/users/03f5d87a43674d67b43082c23c068d83/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:20:20.050768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec6e94-b7c8-4573-b6f8-e05d1d6a48c7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.051108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec6e94-b7c8-4573-b6f8-e05d1d6a48c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.051333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec6e94-b7c8-4573-b6f8-e05d1d6a48c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.078943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-deec6e94-b7c8-4573-b6f8-e05d1d6a48c7 None None] MFA Rules not processed for user `03f5d87a43674d67b43082c23c068d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:20.080173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-deec6e94-b7c8-4573-b6f8-e05d1d6a48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:20.149730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1178/2293] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:20.154160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:20.214693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879de71-def9-4818-8a71-deea4a220b06 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:20.219184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:20.219894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.220142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.220422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.221120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:20.223034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.316295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1116/2294] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:20:20 2022] POST /identity/v3/projects => generated 357 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:20:20.320354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec6e94-b7c8-4573-b6f8-e05d1d6a48c7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:20.332727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec6e94-b7c8-4573-b6f8-e05d1d6a48c7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:20.388236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:20.388821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.389100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.389375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.390066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:20.392022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.460283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1179/2295] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:20:20 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:20:20.461744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:20.472076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d8b17-55cd-404b-93f6-9e21f723ff01 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:20.476604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:20.477269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.477550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.478615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.479434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:20.481103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.491896 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1117/2296] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:20.496447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:20.506390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c946d-c4cc-4376-909d-77579757ebe1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:20.511021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:20.511949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.512890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.513179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] PATH_INFO: `/v3/projects/3d98ca5c29e4430a9b318ca7e77cddfa/users/1b975f789d22481d9de855c4ecf88c31/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.528425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d98ca5c29e4430a9b318ca7e77cddfa, user_id=1b975f789d22481d9de855c4ecf88c31, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:20.531101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.607132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1180/2297] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:20:20 2022] PUT /identity/v3/projects/3d98ca5c29e4430a9b318ca7e77cddfa/users/1b975f789d22481d9de855c4ecf88c31/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:20.611497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:20.622210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78bf8-336a-4d77-97a1-7774819fd85b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:20.683803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743aaee-ab97-48ec-b7d8-b3072ca73445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:20.684382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743aaee-ab97-48ec-b7d8-b3072ca73445 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.684662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743aaee-ab97-48ec-b7d8-b3072ca73445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.684936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743aaee-ab97-48ec-b7d8-b3072ca73445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.685672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743aaee-ab97-48ec-b7d8-b3072ca73445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:20.687312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743aaee-ab97-48ec-b7d8-b3072ca73445 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.700666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1118/2298] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:20.702123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:20.712510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c1181-1f46-4f64-bc45-8d4c0146e80f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:20.717333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:20.718224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.718511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.718791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] PATH_INFO: `/v3/projects/3d98ca5c29e4430a9b318ca7e77cddfa/users/1b975f789d22481d9de855c4ecf88c31/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.720849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d98ca5c29e4430a9b318ca7e77cddfa, user_id=1b975f789d22481d9de855c4ecf88c31, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:20.723511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.734382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1181/2299] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:20:20 2022] PUT /identity/v3/projects/3d98ca5c29e4430a9b318ca7e77cddfa/users/1b975f789d22481d9de855c4ecf88c31/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:20.746611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039afaef-5848-4cda-9ff8-1eb79c5bc98f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.746924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039afaef-5848-4cda-9ff8-1eb79c5bc98f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.747199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039afaef-5848-4cda-9ff8-1eb79c5bc98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.773838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-039afaef-5848-4cda-9ff8-1eb79c5bc98f None None] MFA Rules not processed for user `1b975f789d22481d9de855c4ecf88c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:20.775088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-039afaef-5848-4cda-9ff8-1eb79c5bc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:20.845078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1119/2300] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:20.857780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:20.859499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:20.944051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbf171-2e57-4453-bd01-3d76c53dce7c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:20.947811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567622a-92ec-4491-84f8-82da98cf98c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:20.948458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567622a-92ec-4491-84f8-82da98cf98c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:20.948741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567622a-92ec-4491-84f8-82da98cf98c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:20.949018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567622a-92ec-4491-84f8-82da98cf98c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:20.958112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567622a-92ec-4491-84f8-82da98cf98c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:20.959550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567622a-92ec-4491-84f8-82da98cf98c6 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:20.969949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1182/2301] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:20 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:21.152027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039afaef-5848-4cda-9ff8-1eb79c5bc98f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:21.162790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039afaef-5848-4cda-9ff8-1eb79c5bc98f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:21.169705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:21.170570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:21.171381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:21.171381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:21.249098 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:21.250551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:21.260888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1120/2302] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:21 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:28.068090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:28.068588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:28.068681 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:28.086466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:28.087676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:28.143762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1183/2303] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:20:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:28.284209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:28.285623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:28.353768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d598f-3d2b-44d4-affa-d0197b774bc8 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:28.359027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:28.359768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:28.360055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:28.360333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:28.368865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:28.370448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:28.378979 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1121/2304] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:20:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:20:31.871411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:31.872866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:31.956704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc67214-b347-455f-af33-8dbc66870514 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:31.962855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:31.963522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:31.963848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:31.964124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:31.967152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:32.033425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:32.035157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:32.046883 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1184/2305] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:39.536065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:39.537603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:39.593167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34977380-fdb9-4c94-9fcb-0c6340374215 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:39.596761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23705bcf-62f3-4103-ba17-8c7241fa3908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:39.597405 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23705bcf-62f3-4103-ba17-8c7241fa3908 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:39.597690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23705bcf-62f3-4103-ba17-8c7241fa3908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:39.597968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23705bcf-62f3-4103-ba17-8c7241fa3908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:39.599756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-23705bcf-62f3-4103-ba17-8c7241fa3908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:39.609049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23705bcf-62f3-4103-ba17-8c7241fa3908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:39.610473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23705bcf-62f3-4103-ba17-8c7241fa3908 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:39.618180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1122/2306] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:40.491679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:40.554282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457e786-b773-453f-b80e-c03fe65efba5 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:40.559709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:40.560581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:40.560859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:40.561127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] PATH_INFO: `/v3/users/03f5d87a43674d67b43082c23c068d83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:40.562494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f5d87a43674d67b43082c23c068d83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:40.564583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:40.750979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f5d87a43674d67b43082c23c068d83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:40.828503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f5d87a43674d67b43082c23c068d83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:40.836195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f5d87a43674d67b43082c23c068d83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:40.836856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f5d87a43674d67b43082c23c068d83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:40.849259 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f5d87a43674d67b43082c23c068d83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:40.855861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1185/2307] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:20:40 2022] DELETE /identity/v3/users/03f5d87a43674d67b43082c23c068d83 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:40.860526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:40.860991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:40.861114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:40.878993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:40.887218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:40.937416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1123/2308] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:40.950750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:40.954058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:41.010751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3111-0f9e-493b-b77c-6df786a02d19 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:41.014438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:41.015124 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:41.015401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:41.015703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:41.024567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:41.025989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:41.035727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1186/2309] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:41.476312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:41.477837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:41.490814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef2608-fa85-4b72-ad1a-5a41ab6e9dad None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:41.495415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:41.496929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:41.497206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:41.497838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] PATH_INFO: `/v3/projects/b87bacc307f64fc0be785e3c1da6bcad` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:41.498650 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b87bacc307f64fc0be785e3c1da6bcad)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:41.500684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:41.609283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b87bacc307f64fc0be785e3c1da6bcad', 'request_id': 'req-16753a8e-ba0d-4821-8163-09c48c74ef9b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:41.662902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b87bacc307f64fc0be785e3c1da6bcad', 'request_id': 'req-16753a8e-ba0d-4821-8163-09c48c74ef9b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:41.670115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b87bacc307f64fc0be785e3c1da6bcad', 'request_id': 'req-16753a8e-ba0d-4821-8163-09c48c74ef9b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:41.681093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] The token cache is being invalidate because project b87bacc307f64fc0be785e3c1da6bcad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:20:41.681461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:41.682230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:41.685122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1124/2310] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:20:41 2022] DELETE /identity/v3/projects/b87bacc307f64fc0be785e3c1da6bcad => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:41.689636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:41.699483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020c969-3175-450f-88d9-4ad5e62eb6a5 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:41.755727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:41.756599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:41.756881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:41.757217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] PATH_INFO: `/v3/users/1b975f789d22481d9de855c4ecf88c31` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:41.758333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b975f789d22481d9de855c4ecf88c31)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:41.760281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:41.964171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b975f789d22481d9de855c4ecf88c31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.062903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b975f789d22481d9de855c4ecf88c31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.070714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b975f789d22481d9de855c4ecf88c31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.071371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b975f789d22481d9de855c4ecf88c31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.082734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fe77fe92-f577-465f-a332-34ac34ecf3d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b975f789d22481d9de855c4ecf88c31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.088597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1187/2311] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:20:41 2022] DELETE /identity/v3/users/1b975f789d22481d9de855c4ecf88c31 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:20:42.476295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:42.477827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:42.490868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753a8e-ba0d-4821-8163-09c48c74ef9b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:42.546668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:42.547450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:42.547768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:42.548055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] PATH_INFO: `/v3/projects/3d98ca5c29e4430a9b318ca7e77cddfa` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:42.549233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d98ca5c29e4430a9b318ca7e77cddfa)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:42.551150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:42.671033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d98ca5c29e4430a9b318ca7e77cddfa', 'request_id': 'req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.740201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d98ca5c29e4430a9b318ca7e77cddfa', 'request_id': 'req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.746569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d98ca5c29e4430a9b318ca7e77cddfa', 'request_id': 'req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.757796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] The token cache is being invalidate because project 3d98ca5c29e4430a9b318ca7e77cddfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:20:42.758081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.758722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a5dd8954-c5d6-41de-9fc5-9e9fce00a3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:42.760293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1125/2312] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:20:42 2022] DELETE /identity/v3/projects/3d98ca5c29e4430a9b318ca7e77cddfa => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:42.766060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b6c9-9902-493e-86d5-51a48c834391 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:42.770148 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b6c9-9902-493e-86d5-51a48c834391 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:42.770552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b6c9-9902-493e-86d5-51a48c834391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:42.790422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7289b6c9-9902-493e-86d5-51a48c834391 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:42.791976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7289b6c9-9902-493e-86d5-51a48c834391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:42.856770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1188/2313] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:42.865120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c8407-c829-4427-9b28-4e7912c8a193 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:42.865432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c8407-c829-4427-9b28-4e7912c8a193 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:42.865729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c8407-c829-4427-9b28-4e7912c8a193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:42.885023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5e5c8407-c829-4427-9b28-4e7912c8a193 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:42.887224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5e5c8407-c829-4427-9b28-4e7912c8a193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:42.894941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1126/2314] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:20:42.899535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289b6c9-9902-493e-86d5-51a48c834391 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:42.913197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289b6c9-9902-493e-86d5-51a48c834391 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:42.917801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:42.918625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:42.918903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:42.919180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:42.920040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:20:42.920362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:42.921939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:42.934562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1189/2315] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:20:42.939025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c8407-c829-4427-9b28-4e7912c8a193 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:42.948272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c8407-c829-4427-9b28-4e7912c8a193 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:42.953877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:42.954533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:42.954815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:42.955090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:42.955845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:42.957787 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.030963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1127/2316] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:20:42 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:20:43.035265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.045256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b2560-6799-4bc8-99d3-fe1fa2cc6473 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.102210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.102910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.103189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.103533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.104272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.106993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.172150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1190/2317] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:20:43 2022] POST /identity/v3/users => generated 283 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:20:43.176487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.187209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde88-6086-47cc-afbb-325104e83bcf admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.197099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.198424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.198424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.198424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.199062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.200779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.212453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1128/2318] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:20:43.216896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.227868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9671cf8-1719-4244-af0d-e3c7354fb3a1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.233311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e0ac8-8d19-4f24-a7b7-57cc365d1d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.233997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e0ac8-8d19-4f24-a7b7-57cc365d1d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.234282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e0ac8-8d19-4f24-a7b7-57cc365d1d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.234567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e0ac8-8d19-4f24-a7b7-57cc365d1d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.235297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e0ac8-8d19-4f24-a7b7-57cc365d1d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.236934 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e0ac8-8d19-4f24-a7b7-57cc365d1d7f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.248105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1191/2319] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:43.252477 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.262386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc9a08-9658-4910-94a7-fde00a2c7fd0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.267176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.268170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.268453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.268729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] PATH_INFO: `/v3/projects/35bc3c8de60e490db407c218ca8a286d/users/526ad878e70945d197ffa949dfdae1d7/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.283870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35bc3c8de60e490db407c218ca8a286d, user_id=526ad878e70945d197ffa949dfdae1d7, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.286531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.365910 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1129/2320] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:20:43 2022] PUT /identity/v3/projects/35bc3c8de60e490db407c218ca8a286d/users/526ad878e70945d197ffa949dfdae1d7/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:43.377129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525804af-b6a5-486d-b778-cd5c6db32836 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.377504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525804af-b6a5-486d-b778-cd5c6db32836 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.377843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525804af-b6a5-486d-b778-cd5c6db32836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.404197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-525804af-b6a5-486d-b778-cd5c6db32836 None None] MFA Rules not processed for user `526ad878e70945d197ffa949dfdae1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:43.405378 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-525804af-b6a5-486d-b778-cd5c6db32836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:43.468305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1192/2321] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:20:43.472870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.535925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f5c19-3221-41e2-abc1-a67c51e7d057 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.540674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.541332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.541628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.541890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.542606 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.545295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.631318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1130/2322] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:20:43 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:20:43.635272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525804af-b6a5-486d-b778-cd5c6db32836 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.646609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525804af-b6a5-486d-b778-cd5c6db32836 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.702290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.702971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.703255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.703529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.704256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.706187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.816551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1193/2323] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:20:43 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:20:43.819385 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.829800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f930b-55c7-420b-bf33-d8ed907a25f4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.834518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.835175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.839439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.839439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.839439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.839439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.849410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1131/2324] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:43.853606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.864310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58b765-f920-49aa-93e6-60db6b1cf5ed admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:43.869029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:43.870377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:43.870692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:43.870971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] PATH_INFO: `/v3/projects/4b0d1eedf9c74a0f881f9acd10fe504d/users/87a0c87a1a6c4297932af9ef6bfbf48e/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:43.887353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0d1eedf9c74a0f881f9acd10fe504d, user_id=87a0c87a1a6c4297932af9ef6bfbf48e, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:43.890213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:43.980554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1194/2325] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:20:43 2022] PUT /identity/v3/projects/4b0d1eedf9c74a0f881f9acd10fe504d/users/87a0c87a1a6c4297932af9ef6bfbf48e/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:43.985162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:43.995528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fed67-563e-4f40-9d5b-0ab4641a5f68 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:44.052223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72849426-799f-4ddd-a590-edec9be0c64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:44.052893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72849426-799f-4ddd-a590-edec9be0c64d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:44.053175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72849426-799f-4ddd-a590-edec9be0c64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:44.053453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72849426-799f-4ddd-a590-edec9be0c64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:44.054175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72849426-799f-4ddd-a590-edec9be0c64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:44.055809 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72849426-799f-4ddd-a590-edec9be0c64d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:44.065961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1132/2326] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:44.074079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:44.085181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5358ae3-5377-415d-93c0-6bc6eb6579dc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:44.090599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:44.091473 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:44.091802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:44.092079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] PATH_INFO: `/v3/projects/4b0d1eedf9c74a0f881f9acd10fe504d/users/87a0c87a1a6c4297932af9ef6bfbf48e/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:44.094106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0d1eedf9c74a0f881f9acd10fe504d, user_id=87a0c87a1a6c4297932af9ef6bfbf48e, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:44.096845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:44.106742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1195/2327] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:20:44 2022] PUT /identity/v3/projects/4b0d1eedf9c74a0f881f9acd10fe504d/users/87a0c87a1a6c4297932af9ef6bfbf48e/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:44.119061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02190-10a1-4db1-99a7-1c8414c925c6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:44.119355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02190-10a1-4db1-99a7-1c8414c925c6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:44.119659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02190-10a1-4db1-99a7-1c8414c925c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:44.147502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-66c02190-10a1-4db1-99a7-1c8414c925c6 None None] MFA Rules not processed for user `87a0c87a1a6c4297932af9ef6bfbf48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:44.148662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-66c02190-10a1-4db1-99a7-1c8414c925c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:44.216785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1133/2328] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:44.228111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:44.229811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:44.284799 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cd7f7-82b5-4884-b43f-257c54844b8e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:44.288415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a24d58-be3f-4161-9979-6c4930dfff21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:44.289056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a24d58-be3f-4161-9979-6c4930dfff21 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:44.289337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a24d58-be3f-4161-9979-6c4930dfff21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:44.289612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a24d58-be3f-4161-9979-6c4930dfff21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:44.299351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a24d58-be3f-4161-9979-6c4930dfff21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:44.300826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a24d58-be3f-4161-9979-6c4930dfff21 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:44.310538 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1196/2329] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:44.611221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c02190-10a1-4db1-99a7-1c8414c925c6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:44.620559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c02190-10a1-4db1-99a7-1c8414c925c6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:44.625312 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecdd0c-6c44-4fe5-af1d-0b4697cc0f9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:44.625671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecdd0c-6c44-4fe5-af1d-0b4697cc0f9a service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:44.625951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecdd0c-6c44-4fe5-af1d-0b4697cc0f9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:44.626255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecdd0c-6c44-4fe5-af1d-0b4697cc0f9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:44.682668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecdd0c-6c44-4fe5-af1d-0b4697cc0f9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:44.685516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecdd0c-6c44-4fe5-af1d-0b4697cc0f9a service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:44.694119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1134/2330] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:45.102625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41dca0-d467-458a-958e-188278a230cf None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.103078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41dca0-d467-458a-958e-188278a230cf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.103320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41dca0-d467-458a-958e-188278a230cf None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.104977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1197/2331] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:20:45.113460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483d1e-5c9c-453a-9e70-c20ea4663e1e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.113898 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483d1e-5c9c-453a-9e70-c20ea4663e1e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.114185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483d1e-5c9c-453a-9e70-c20ea4663e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.135655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-90483d1e-5c9c-453a-9e70-c20ea4663e1e None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:45.137577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-90483d1e-5c9c-453a-9e70-c20ea4663e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:45.204507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1135/2332] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:45.216863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41dca0-d467-458a-958e-188278a230cf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:45.218513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bd41dca0-d467-458a-958e-188278a230cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:45.276250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41dca0-d467-458a-958e-188278a230cf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:45.280265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d368f-5c56-47f1-adae-795bbc10b72c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:45.280912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d368f-5c56-47f1-adae-795bbc10b72c service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.281195 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d368f-5c56-47f1-adae-795bbc10b72c service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.281472 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d368f-5c56-47f1-adae-795bbc10b72c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.291506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d368f-5c56-47f1-adae-795bbc10b72c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:45.293010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d368f-5c56-47f1-adae-795bbc10b72c service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:45.304311 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1198/2333] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:45.564726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d279b2-d5bf-4224-8be0-f8bd8dcfde0a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.564934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d279b2-d5bf-4224-8be0-f8bd8dcfde0a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.565183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d279b2-d5bf-4224-8be0-f8bd8dcfde0a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.566589 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1136/2334] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:20:45.571387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f753-b830-4d26-bed5-e368b8c7f1c0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.571756 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f753-b830-4d26-bed5-e368b8c7f1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.572040 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f753-b830-4d26-bed5-e368b8c7f1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.609668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4253f753-b830-4d26-bed5-e368b8c7f1c0 None None] MFA Rules not processed for user `76b2a2d381a14f5fb04e721905b36ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:45.611062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4253f753-b830-4d26-bed5-e368b8c7f1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:45.661515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1199/2335] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:45.674010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d279b2-d5bf-4224-8be0-f8bd8dcfde0a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:45.675807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a1d279b2-d5bf-4224-8be0-f8bd8dcfde0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:45.689545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d279b2-d5bf-4224-8be0-f8bd8dcfde0a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:45.693021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113a676-66d9-40e8-bd64-83db7fe121b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:45.693649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113a676-66d9-40e8-bd64-83db7fe121b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.693925 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113a676-66d9-40e8-bd64-83db7fe121b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.694194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113a676-66d9-40e8-bd64-83db7fe121b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.701783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113a676-66d9-40e8-bd64-83db7fe121b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:45.703253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113a676-66d9-40e8-bd64-83db7fe121b9 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:45.710668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1137/2336] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:45.824167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89eaee-8b73-4e4c-be96-671d00995685 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.824529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89eaee-8b73-4e4c-be96-671d00995685 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.824826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89eaee-8b73-4e4c-be96-671d00995685 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.826227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1200/2337] 10.222.0.71 () {54 vars in 921 bytes} [Mon May 16 15:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 15:20:45.831614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.832262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.832540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.851146 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] MFA Rules not processed for user `3d58a6e2eac047ad8e88a1f973ecc697`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:45.852357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:45.862145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1138/2338] 10.222.0.71 () {58 vars in 1018 bytes} [Mon May 16 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:45.872405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89eaee-8b73-4e4c-be96-671d00995685 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:45.874091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1e89eaee-8b73-4e4c-be96-671d00995685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:45.887569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89eaee-8b73-4e4c-be96-671d00995685 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:45.891506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d13605-13f4-4fa6-a2be-f99372a5b5f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:45.892197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13605-13f4-4fa6-a2be-f99372a5b5f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:45.892477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13605-13f4-4fa6-a2be-f99372a5b5f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:45.892749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13605-13f4-4fa6-a2be-f99372a5b5f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:45.903429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d13605-13f4-4fa6-a2be-f99372a5b5f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:45.904872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d13605-13f4-4fa6-a2be-f99372a5b5f8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:45.915470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1201/2339] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:45.974595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:45.975969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:46.043406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dac0a-a431-42c9-9932-9968d04edcce None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:46.048413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:46.049055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:46.049335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:46.049608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:46.051365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:46.062585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:46.064190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:46.073609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1139/2340] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:48.502967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94b17-6dce-4fc4-9a4e-3beba8cd67e8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:48.502967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94b17-6dce-4fc4-9a4e-3beba8cd67e8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:48.502967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94b17-6dce-4fc4-9a4e-3beba8cd67e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:48.520360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-34e94b17-6dce-4fc4-9a4e-3beba8cd67e8 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:48.521831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-34e94b17-6dce-4fc4-9a4e-3beba8cd67e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:48.578192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1202/2341] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:20:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:48.590986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:48.594277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:48.663476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca67d3-9db3-468e-be07-4e0f9445e4d4 service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:48.669637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457daa7-dc91-4342-ac8f-ca39ea739ffa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:48.670296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457daa7-dc91-4342-ac8f-ca39ea739ffa service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:48.670576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457daa7-dc91-4342-ac8f-ca39ea739ffa service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:48.670902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457daa7-dc91-4342-ac8f-ca39ea739ffa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:48.679432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457daa7-dc91-4342-ac8f-ca39ea739ffa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:48.681074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457daa7-dc91-4342-ac8f-ca39ea739ffa service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:48.690269 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1140/2342] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:20:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:20:57.018496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94b17-6dce-4fc4-9a4e-3beba8cd67e8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:57.124984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94b17-6dce-4fc4-9a4e-3beba8cd67e8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:57.130009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:57.131172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:57.131647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:57.132091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] PATH_INFO: `/v3/users/526ad878e70945d197ffa949dfdae1d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:57.133608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=526ad878e70945d197ffa949dfdae1d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:57.135813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:57.372280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526ad878e70945d197ffa949dfdae1d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:57.454444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526ad878e70945d197ffa949dfdae1d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:57.461498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526ad878e70945d197ffa949dfdae1d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:57.462132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526ad878e70945d197ffa949dfdae1d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:57.474304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526ad878e70945d197ffa949dfdae1d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:57.480152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1203/2343] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:20:57 2022] DELETE /identity/v3/users/526ad878e70945d197ffa949dfdae1d7 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:57.485483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:57.486069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:57.486185 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:57.503604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:57.504834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:57.560836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1141/2344] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:57.570243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:57.571647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:57.627281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e48b8-f176-408c-8a14-24581e6a95e4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:57.630897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:57.631535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:57.631850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:57.632129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:57.640970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:57.642401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:57.651968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1204/2345] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:20:58.047163 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:58.051668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:58.060561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3d46d-d342-4a16-bb10-ca08286a2829 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:58.065194 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:58.066059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:58.066337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:58.066610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] PATH_INFO: `/v3/projects/35bc3c8de60e490db407c218ca8a286d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:58.067857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35bc3c8de60e490db407c218ca8a286d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:58.069827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:58.182395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35bc3c8de60e490db407c218ca8a286d', 'request_id': 'req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.257472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35bc3c8de60e490db407c218ca8a286d', 'request_id': 'req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.263741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35bc3c8de60e490db407c218ca8a286d', 'request_id': 'req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.274727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] The token cache is being invalidate because project 35bc3c8de60e490db407c218ca8a286d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:20:58.275010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.275646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.277197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1142/2346] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:20:58 2022] DELETE /identity/v3/projects/35bc3c8de60e490db407c218ca8a286d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:58.281200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:58.291092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dbdaf-abfd-4218-ae1c-8830d7a12f30 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:58.349135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:58.350147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:58.350425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:58.350695 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] PATH_INFO: `/v3/users/87a0c87a1a6c4297932af9ef6bfbf48e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:58.352351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a0c87a1a6c4297932af9ef6bfbf48e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:58.354267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:58.556559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a0c87a1a6c4297932af9ef6bfbf48e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.628521 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a0c87a1a6c4297932af9ef6bfbf48e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.635761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a0c87a1a6c4297932af9ef6bfbf48e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.637688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a0c87a1a6c4297932af9ef6bfbf48e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.649029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9b174092-91b7-4eef-917a-1c71cbf36791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a0c87a1a6c4297932af9ef6bfbf48e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:58.655484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1205/2347] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:20:58 2022] DELETE /identity/v3/users/87a0c87a1a6c4297932af9ef6bfbf48e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:59.055951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:59.057379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:59.071371 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23549d9-ee0f-48aa-8d41-1f20bb8566d8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:59.128478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:59.129332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.129611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.129880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] PATH_INFO: `/v3/projects/4b0d1eedf9c74a0f881f9acd10fe504d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.131112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0d1eedf9c74a0f881f9acd10fe504d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:59.133043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:59.295541 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0d1eedf9c74a0f881f9acd10fe504d', 'request_id': 'req-8448a5e2-0235-4483-b074-7ccc9ab4da21'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:59.363196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0d1eedf9c74a0f881f9acd10fe504d', 'request_id': 'req-8448a5e2-0235-4483-b074-7ccc9ab4da21'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:59.368972 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0d1eedf9c74a0f881f9acd10fe504d', 'request_id': 'req-8448a5e2-0235-4483-b074-7ccc9ab4da21'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:59.380879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] The token cache is being invalidate because project 4b0d1eedf9c74a0f881f9acd10fe504d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:20:59.381169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:59.381807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8448a5e2-0235-4483-b074-7ccc9ab4da21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:20:59.383387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1143/2348] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:20:59 2022] DELETE /identity/v3/projects/4b0d1eedf9c74a0f881f9acd10fe504d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:20:59.388190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc5982-4937-418d-abf5-af45d85eed6d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.388793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc5982-4937-418d-abf5-af45d85eed6d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.389482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc5982-4937-418d-abf5-af45d85eed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.407508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-11dc5982-4937-418d-abf5-af45d85eed6d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:59.408794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-11dc5982-4937-418d-abf5-af45d85eed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:59.469419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1206/2349] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:59.477148 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a976-432a-4c33-b8f1-98ce29e4548e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.477515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a976-432a-4c33-b8f1-98ce29e4548e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.477865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31a976-432a-4c33-b8f1-98ce29e4548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.497050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6e31a976-432a-4c33-b8f1-98ce29e4548e None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:20:59.498939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6e31a976-432a-4c33-b8f1-98ce29e4548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:20:59.507276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1144/2350] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:20:59.511373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc5982-4937-418d-abf5-af45d85eed6d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:59.522742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc5982-4937-418d-abf5-af45d85eed6d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:59.527323 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:59.528016 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.531235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.531235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.531235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:20:59.531235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:59.531802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:59.543241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1207/2351] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:20:59.548299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31a976-432a-4c33-b8f1-98ce29e4548e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:59.557418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31a976-432a-4c33-b8f1-98ce29e4548e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:59.562798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:59.563459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.563775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.564053 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.564757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:59.566651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:59.670735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1145/2352] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:20:59 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:20:59.674730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:59.684605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666e288-a7a0-44c6-97a6-9ec766338b81 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:59.740025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:59.740688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.740968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.741241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.741923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:59.743848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:59.820163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1208/2353] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:20:59 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:20:59.824398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:59.835043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0631df5-61c2-433e-baf5-da7c0b161eb7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:59.839622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:59.840563 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.840821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.841103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.842348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:59.843994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:59.854579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1146/2354] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:59.859659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:59.869527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69961a98-2f8b-431d-a24b-94cef5a157b1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:59.874730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c94f48-2c16-4ac1-bdcd-08179fbd78d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:59.875995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c94f48-2c16-4ac1-bdcd-08179fbd78d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.875995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c94f48-2c16-4ac1-bdcd-08179fbd78d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.876190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c94f48-2c16-4ac1-bdcd-08179fbd78d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.876905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c94f48-2c16-4ac1-bdcd-08179fbd78d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:59.878555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c94f48-2c16-4ac1-bdcd-08179fbd78d3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:20:59.888951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1209/2355] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:20:59.893007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:20:59.903090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34936b08-3cc6-4f89-b576-cdec14c5305b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:20:59.908425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641521-1138-4657-b886-e801740c880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:20:59.909300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641521-1138-4657-b886-e801740c880e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:20:59.909611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641521-1138-4657-b886-e801740c880e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:20:59.909886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641521-1138-4657-b886-e801740c880e admin admin] PATH_INFO: `/v3/projects/3a4d04c737ba49268cde2b254f5cf7de/users/265da752a6734161aa7ab5e1ffb62633/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:20:59.924410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e641521-1138-4657-b886-e801740c880e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4d04c737ba49268cde2b254f5cf7de, user_id=265da752a6734161aa7ab5e1ffb62633, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:20:59.927075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e641521-1138-4657-b886-e801740c880e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.017233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1147/2356] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:20:59 2022] PUT /identity/v3/projects/3a4d04c737ba49268cde2b254f5cf7de/users/265da752a6734161aa7ab5e1ffb62633/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:00.030135 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c56005-a0f8-4396-afbd-89cbd26e06f9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.030395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c56005-a0f8-4396-afbd-89cbd26e06f9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.031028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c56005-a0f8-4396-afbd-89cbd26e06f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.057341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-02c56005-a0f8-4396-afbd-89cbd26e06f9 None None] MFA Rules not processed for user `265da752a6734161aa7ab5e1ffb62633`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:00.058558 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02c56005-a0f8-4396-afbd-89cbd26e06f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:00.121084 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1210/2357] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:00.125647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641521-1138-4657-b886-e801740c880e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:00.185651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641521-1138-4657-b886-e801740c880e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:00.190305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:00.190950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.191231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.191502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.192216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:00.194099 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.281651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1148/2358] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:00 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:00.285690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c56005-a0f8-4396-afbd-89cbd26e06f9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:00.296067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c56005-a0f8-4396-afbd-89cbd26e06f9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:00.351491 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:00.352470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.352767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.353047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.353730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:00.355690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.428255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1211/2359] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:00 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:00.431951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:00.442804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b977cd-b5e0-43c0-8aaa-08669ab53a05 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:00.447283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:00.447963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.448245 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.448510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.449232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:00.450926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.461902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1149/2360] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:00.465767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:00.475536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768ace3-2dbf-4ad0-b6f0-4b125c71cd2a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:00.480077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:00.480924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.481222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.481478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] PATH_INFO: `/v3/projects/40fe41aa75d24a4f8ffaa9e70c3e955e/users/9e42a823be0c4c0cac72eeed0513ecd9/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.495286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fe41aa75d24a4f8ffaa9e70c3e955e, user_id=9e42a823be0c4c0cac72eeed0513ecd9, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:00.497957 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.585208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1212/2361] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:00 2022] PUT /identity/v3/projects/40fe41aa75d24a4f8ffaa9e70c3e955e/users/9e42a823be0c4c0cac72eeed0513ecd9/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:00.589188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:00.599503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a15cf-c1ce-480f-9b16-5def5cac7d55 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:00.657361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539a065-32df-4d0d-bbe8-fa870b65fb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:00.658033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539a065-32df-4d0d-bbe8-fa870b65fb1d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.658317 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539a065-32df-4d0d-bbe8-fa870b65fb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.658595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539a065-32df-4d0d-bbe8-fa870b65fb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.659341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539a065-32df-4d0d-bbe8-fa870b65fb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:00.660975 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539a065-32df-4d0d-bbe8-fa870b65fb1d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.670639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1150/2362] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:00.674897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:00.685156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfb876-e8a4-450d-a7ab-4d0c101e2aa1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:00.690373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:00.691253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.691534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.691856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] PATH_INFO: `/v3/projects/40fe41aa75d24a4f8ffaa9e70c3e955e/users/9e42a823be0c4c0cac72eeed0513ecd9/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.693789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fe41aa75d24a4f8ffaa9e70c3e955e, user_id=9e42a823be0c4c0cac72eeed0513ecd9, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:00.696494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.707967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1213/2363] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:00 2022] PUT /identity/v3/projects/40fe41aa75d24a4f8ffaa9e70c3e955e/users/9e42a823be0c4c0cac72eeed0513ecd9/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:00.719073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.719073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.719289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.746691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] MFA Rules not processed for user `9e42a823be0c4c0cac72eeed0513ecd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:00.747938 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:00.816520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1151/2364] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:00.826939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:00.828700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:00.889366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99721b8-e673-4664-9297-4b266349a1ee admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:00.893453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:00.894100 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:00.894382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:00.894660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:00.951504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:00.953011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:00.961989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1214/2365] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:02.961448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:02.962918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:03.035439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f92b71-2a40-44c8-b720-da0d3316dcfc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:03.040861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47252cb-3261-4d04-8439-c177ef372d70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:03.041517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47252cb-3261-4d04-8439-c177ef372d70 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:03.041799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47252cb-3261-4d04-8439-c177ef372d70 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:03.042128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47252cb-3261-4d04-8439-c177ef372d70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:03.043991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d47252cb-3261-4d04-8439-c177ef372d70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:03.061588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47252cb-3261-4d04-8439-c177ef372d70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:03.063967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47252cb-3261-4d04-8439-c177ef372d70 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:03.081377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1152/2366] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:04.873619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:04.875048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:04.891334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e723dbc-4b34-423d-9e89-476f9e390576 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:04.897036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:04.897666 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:04.897950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:04.898225 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:04.899946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:04.957154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:04.958927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:04.967997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1215/2367] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:07.596753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd3dc-8e11-4434-9822-32e9c72567d7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:07.597265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd3dc-8e11-4434-9822-32e9c72567d7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:07.597357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd3dc-8e11-4434-9822-32e9c72567d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:07.615483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a9bdd3dc-8e11-4434-9822-32e9c72567d7 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:07.616690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a9bdd3dc-8e11-4434-9822-32e9c72567d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:07.671795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1153/2368] 10.222.0.71 () {58 vars in 967 bytes} [Mon May 16 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:07.683776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:07.685169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:07.768783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc572e-16d9-424b-a878-0205db245f0d service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:07.774268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:07.775044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:07.775397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:07.776219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:07.788485 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:07.790111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:07.798660 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1216/2369] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:21:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:21:08.179768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd3dc-8e11-4434-9822-32e9c72567d7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:08.188719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd3dc-8e11-4434-9822-32e9c72567d7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:08.193026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:08.194639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:08.194639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:08.194639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:08.203281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:08.204735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:08.214996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1154/2370] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:27.279078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:27.289393 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cbd89-caf4-4557-ba8e-c67b19744038 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:27.292946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:27.293576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:27.293855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:27.294128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:27.295906 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:27.355399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:27.356846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:27.366832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1217/2371] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:34.734445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:34.747401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e310a46-d936-46fb-b247-26aada79915b service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:34.752927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c17a68-a03e-4d1c-8251-9617ba2b4fda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:34.753737 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c17a68-a03e-4d1c-8251-9617ba2b4fda service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:34.754027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c17a68-a03e-4d1c-8251-9617ba2b4fda service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:34.754310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c17a68-a03e-4d1c-8251-9617ba2b4fda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:34.756673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-87c17a68-a03e-4d1c-8251-9617ba2b4fda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:34.769239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c17a68-a03e-4d1c-8251-9617ba2b4fda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:34.770847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c17a68-a03e-4d1c-8251-9617ba2b4fda service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:34.782834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1155/2372] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:35.060063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:35.147987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6208a46-8a46-4065-9b32-cf2cbceb7404 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:35.150303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:35.151430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:35.151758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:35.152035 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] PATH_INFO: `/v3/users/265da752a6734161aa7ab5e1ffb62633` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:35.153243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] RBAC: Authorizing `identity:delete_user(user_id=265da752a6734161aa7ab5e1ffb62633)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:35.155217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:35.341811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '265da752a6734161aa7ab5e1ffb62633'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:35.434528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '265da752a6734161aa7ab5e1ffb62633'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:35.442911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '265da752a6734161aa7ab5e1ffb62633'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:35.444604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '265da752a6734161aa7ab5e1ffb62633'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:35.457842 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '265da752a6734161aa7ab5e1ffb62633'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:35.465063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1218/2373] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:35 2022] DELETE /identity/v3/users/265da752a6734161aa7ab5e1ffb62633 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:35.469547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:35.472636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:35.472636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:35.502243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:35.503573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:35.568116 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1156/2374] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:35.577849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:35.579200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:35.650253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee28c2-9df6-4fa7-99a9-531e9de8814d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:35.654077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:35.654758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:35.655039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:35.655314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:35.666009 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:35.667726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:35.679586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1219/2375] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:36.078891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:36.080319 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:36.093513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc408a5-e44d-4209-b612-4ade8ae5b9c8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:36.098189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:36.099064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:36.099379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:36.099683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] PATH_INFO: `/v3/projects/3a4d04c737ba49268cde2b254f5cf7de` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:36.100841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4d04c737ba49268cde2b254f5cf7de)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:36.102783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:36.216622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4d04c737ba49268cde2b254f5cf7de', 'request_id': 'req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.296812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4d04c737ba49268cde2b254f5cf7de', 'request_id': 'req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.303012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4d04c737ba49268cde2b254f5cf7de', 'request_id': 'req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.313805 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] The token cache is being invalidate because project 3a4d04c737ba49268cde2b254f5cf7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:36.314092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.314708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.316256 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1157/2376] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:36 2022] DELETE /identity/v3/projects/3a4d04c737ba49268cde2b254f5cf7de => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:36.320828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:36.330882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94190d-c411-44d3-b261-98fd7a2f41d8 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:36.385613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:36.386476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:36.386757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:36.387026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] PATH_INFO: `/v3/users/9e42a823be0c4c0cac72eeed0513ecd9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:36.388217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e42a823be0c4c0cac72eeed0513ecd9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:36.390267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:36.581950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e42a823be0c4c0cac72eeed0513ecd9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.637425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e42a823be0c4c0cac72eeed0513ecd9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.644895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e42a823be0c4c0cac72eeed0513ecd9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.645528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e42a823be0c4c0cac72eeed0513ecd9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.657601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d1783c43-c501-4230-bc03-62fab4cdf175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e42a823be0c4c0cac72eeed0513ecd9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:36.663833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1220/2377] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:36 2022] DELETE /identity/v3/users/9e42a823be0c4c0cac72eeed0513ecd9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:37.066696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:37.068162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:37.081069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97cc1f-b62e-4e5c-ad64-b28d5385a210 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:37.136772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:37.139216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.139216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.139216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] PATH_INFO: `/v3/projects/40fe41aa75d24a4f8ffaa9e70c3e955e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.139478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fe41aa75d24a4f8ffaa9e70c3e955e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:37.141283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:37.245087 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fe41aa75d24a4f8ffaa9e70c3e955e', 'request_id': 'req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:37.337270 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fe41aa75d24a4f8ffaa9e70c3e955e', 'request_id': 'req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:37.343321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fe41aa75d24a4f8ffaa9e70c3e955e', 'request_id': 'req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:37.354776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] The token cache is being invalidate because project 40fe41aa75d24a4f8ffaa9e70c3e955e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:37.355002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:37.355662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b58ac26d-9b69-4162-a8c4-d2fdcec9304d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:37.357212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1158/2378] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:37 2022] DELETE /identity/v3/projects/40fe41aa75d24a4f8ffaa9e70c3e955e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:37.362082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9ac71-e128-48be-bf83-3adf94990078 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.362546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9ac71-e128-48be-bf83-3adf94990078 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.362637 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9ac71-e128-48be-bf83-3adf94990078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.382083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e6b9ac71-e128-48be-bf83-3adf94990078 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:37.383283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e6b9ac71-e128-48be-bf83-3adf94990078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:37.441405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1221/2379] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:37.451733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c621f65-9a08-4336-afcc-62cd8ade4553 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.452030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c621f65-9a08-4336-afcc-62cd8ade4553 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.452311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c621f65-9a08-4336-afcc-62cd8ade4553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.470072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3c621f65-9a08-4336-afcc-62cd8ade4553 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:37.471354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3c621f65-9a08-4336-afcc-62cd8ade4553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:37.479436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1159/2380] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:37.483376 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9ac71-e128-48be-bf83-3adf94990078 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:37.494758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9ac71-e128-48be-bf83-3adf94990078 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:37.500042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:37.500751 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.501093 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.501972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.502765 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:21:37.503071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:37.504954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:37.519729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1222/2381] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:21:37.523179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c621f65-9a08-4336-afcc-62cd8ade4553 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:37.533068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c621f65-9a08-4336-afcc-62cd8ade4553 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:37.538689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:37.538796 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.539228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.539955 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.540188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:37.542248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:37.639566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1160/2382] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:37 2022] POST /identity/v3/projects => generated 407 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:37.649667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:37.659664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2af20-c4bb-43c6-93b2-05b451ea7392 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:37.716563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:37.717233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.717519 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.717793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.718484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:37.720495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:37.800430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1223/2383] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:37 2022] POST /identity/v3/users => generated 304 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:37.801853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:37.812037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd55f33-524f-4c61-9658-68b571d919ab admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:37.816708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:37.817365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.817644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.818499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.819221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:37.820867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:37.832128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1161/2384] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:37.836855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:37.847168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48f204-c43a-49d4-95a4-8b5d13c61bff admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:37.852141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19eaec-b096-4cc1-a09e-8a584cd339a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:37.852882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19eaec-b096-4cc1-a09e-8a584cd339a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.853166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19eaec-b096-4cc1-a09e-8a584cd339a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.853440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19eaec-b096-4cc1-a09e-8a584cd339a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.854193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19eaec-b096-4cc1-a09e-8a584cd339a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:37.855847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19eaec-b096-4cc1-a09e-8a584cd339a7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:37.866448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1224/2385] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:21:37.876488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:37.890472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a743-038c-4ab3-b62e-02c459991aa4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:37.897372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:37.898648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:37.898744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:37.899129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] PATH_INFO: `/v3/projects/53d83251ce474848bec4fe4e572698ea/users/4ad85681f27d4ae69db0deafae9ade15/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:37.916420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d83251ce474848bec4fe4e572698ea, user_id=4ad85681f27d4ae69db0deafae9ade15, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:37.919200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:38.032984 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1162/2386] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:37 2022] PUT /identity/v3/projects/53d83251ce474848bec4fe4e572698ea/users/4ad85681f27d4ae69db0deafae9ade15/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:38.034605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5196-e78f-49c6-a2f6-a6155434fea5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.034900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5196-e78f-49c6-a2f6-a6155434fea5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.035181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5196-e78f-49c6-a2f6-a6155434fea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.063646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-22ac5196-e78f-49c6-a2f6-a6155434fea5 None None] MFA Rules not processed for user `4ad85681f27d4ae69db0deafae9ade15`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:38.064874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22ac5196-e78f-49c6-a2f6-a6155434fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:38.189147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1225/2387] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:38.194290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:38.282379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5a66c-c347-4442-8d91-e92073d26207 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:38.293464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:38.296876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.296876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.296876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.296876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:38.298874 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:38.388356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1163/2388] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:38 2022] POST /identity/v3/projects => generated 409 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:38.393317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac5196-e78f-49c6-a2f6-a6155434fea5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:38.407005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac5196-e78f-49c6-a2f6-a6155434fea5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:38.464750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:38.465475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.465785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.466061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.466762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:38.468733 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:38.567654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1226/2389] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:38 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:38.569045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:38.580601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb5ccc-8daa-45fe-b08d-0e7241974f0e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:38.585776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:38.586479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.586762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.587038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.587824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:38.589496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:38.602178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1164/2390] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:38.609060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:38.620402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ed6fb-d0a6-432d-bccf-b6c41da09555 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:38.625525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:38.626419 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.626708 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.626985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] PATH_INFO: `/v3/projects/37bccc1c6be74270967f81536d17f3d3/users/6908c2423da64d459f5819c9efc59980/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.647333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bccc1c6be74270967f81536d17f3d3, user_id=6908c2423da64d459f5819c9efc59980, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:38.650153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:38.743874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1227/2391] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:38 2022] PUT /identity/v3/projects/37bccc1c6be74270967f81536d17f3d3/users/6908c2423da64d459f5819c9efc59980/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:38.748231 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:38.765476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6173ac-657e-44bf-9bd7-4f98e89decdc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:38.830764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568e13a-989e-41f7-a913-7332147541ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:38.831445 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568e13a-989e-41f7-a913-7332147541ff admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.831821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568e13a-989e-41f7-a913-7332147541ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.832277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568e13a-989e-41f7-a913-7332147541ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.833100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568e13a-989e-41f7-a913-7332147541ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:38.835392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568e13a-989e-41f7-a913-7332147541ff admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:38.862328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1165/2392] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:38.863680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:38.877990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4413-bd23-41ad-871b-8dbbcf155289 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:38.891082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:38.892689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.893180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.893647 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] PATH_INFO: `/v3/projects/37bccc1c6be74270967f81536d17f3d3/users/6908c2423da64d459f5819c9efc59980/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.896997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bccc1c6be74270967f81536d17f3d3, user_id=6908c2423da64d459f5819c9efc59980, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:38.900025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:38.919397 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1228/2393] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:38 2022] PUT /identity/v3/projects/37bccc1c6be74270967f81536d17f3d3/users/6908c2423da64d459f5819c9efc59980/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:38.930630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373c086-2747-47e7-ba32-d3c592672678 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:38.930960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373c086-2747-47e7-ba32-d3c592672678 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:38.931203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373c086-2747-47e7-ba32-d3c592672678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:38.963349 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f373c086-2747-47e7-ba32-d3c592672678 None None] MFA Rules not processed for user `6908c2423da64d459f5819c9efc59980`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:38.964647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f373c086-2747-47e7-ba32-d3c592672678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:39.053203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1166/2394] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:39.065553 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:39.067287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:39.141651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37ec9-eece-4577-9e55-02ef2a2e5dc2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:39.145377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:39.150091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:39.150091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:39.150091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:39.157676 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:39.159253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:39.170274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1229/2395] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:39.564568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373c086-2747-47e7-ba32-d3c592672678 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:39.574895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373c086-2747-47e7-ba32-d3c592672678 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:39.578454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:39.579128 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:39.579416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:39.579720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:39.643313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:39.644829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:39.655452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1167/2396] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:39 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:42.838116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:42.839607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:42.894901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268e355-4f5b-45b8-a0bf-22fdf3855f2a service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:42.899243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:42.899902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:42.900181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:42.900465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:42.902172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:42.962415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:42.963879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:42.973541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1230/2397] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:46.864775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:46.866350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:46.881888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc962fc9-649a-4ddc-a106-61513e5be636 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:46.887954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:46.889089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:46.889586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:46.889863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] PATH_INFO: `/v3/users/d38cdf40e33b47d8aa79e029cded7503` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:46.891059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38cdf40e33b47d8aa79e029cded7503)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:46.893036 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:47.065204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38cdf40e33b47d8aa79e029cded7503'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.139983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38cdf40e33b47d8aa79e029cded7503'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.139983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38cdf40e33b47d8aa79e029cded7503'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.152399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38cdf40e33b47d8aa79e029cded7503'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.160028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38cdf40e33b47d8aa79e029cded7503'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.170801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1168/2398] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:46 2022] DELETE /identity/v3/users/d38cdf40e33b47d8aa79e029cded7503 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:47.231007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:47.239699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:47.244568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e6e5d-e96b-4a6f-852e-212d72825fb5 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:47.300644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae7079-7eb0-4d00-8989-86e21446149c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:47.301771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae7079-7eb0-4d00-8989-86e21446149c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:47.302281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae7079-7eb0-4d00-8989-86e21446149c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:47.302796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae7079-7eb0-4d00-8989-86e21446149c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:47.306010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8aae7079-7eb0-4d00-8989-86e21446149c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:47.369365 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae7079-7eb0-4d00-8989-86e21446149c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:47.370820 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae7079-7eb0-4d00-8989-86e21446149c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:47.380421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1231/2399] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:47.507738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:47.509647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:47.527672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e017d7-f6be-4dee-81db-e384e96652ec admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:47.533131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:47.534114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:47.535331 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:47.535993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] PATH_INFO: `/v3/projects/56ba8d3ade6e4efd90ce4c340464c565` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:47.537974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ba8d3ade6e4efd90ce4c340464c565)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:47.540476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:47.655474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ba8d3ade6e4efd90ce4c340464c565', 'request_id': 'req-0bee8886-8f93-4e19-8c1d-97c581831c98'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.725819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ba8d3ade6e4efd90ce4c340464c565', 'request_id': 'req-0bee8886-8f93-4e19-8c1d-97c581831c98'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.732069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ba8d3ade6e4efd90ce4c340464c565', 'request_id': 'req-0bee8886-8f93-4e19-8c1d-97c581831c98'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.743286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] The token cache is being invalidate because project 56ba8d3ade6e4efd90ce4c340464c565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:47.743580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.744262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bee8886-8f93-4e19-8c1d-97c581831c98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:47.745803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1169/2400] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:47 2022] DELETE /identity/v3/projects/56ba8d3ade6e4efd90ce4c340464c565 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:47.751030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e069ed-3438-4103-9943-5649f4566f29 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:47.751333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e069ed-3438-4103-9943-5649f4566f29 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:47.751745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e069ed-3438-4103-9943-5649f4566f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:47.769956 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a0e069ed-3438-4103-9943-5649f4566f29 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:47.771525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a0e069ed-3438-4103-9943-5649f4566f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:47.831024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1232/2401] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:47.839761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a44e8-eb4f-49da-9d62-22e1cb3f9d28 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:47.840056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a44e8-eb4f-49da-9d62-22e1cb3f9d28 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:47.840348 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a44e8-eb4f-49da-9d62-22e1cb3f9d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:47.859741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-508a44e8-eb4f-49da-9d62-22e1cb3f9d28 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:47.860941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-508a44e8-eb4f-49da-9d62-22e1cb3f9d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:47.870455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1170/2402] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:47.874850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e069ed-3438-4103-9943-5649f4566f29 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:47.885154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e069ed-3438-4103-9943-5649f4566f29 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:47.889719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:47.890460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:47.891133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:47.891417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:47.892503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:21:47.892809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:47.894433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:47.906210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1233/2403] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:21:47.910802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a44e8-eb4f-49da-9d62-22e1cb3f9d28 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:47.920638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a44e8-eb4f-49da-9d62-22e1cb3f9d28 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:47.926114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:47.926793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:47.927078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:47.927355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:47.928089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:47.930621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.044548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1171/2404] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:47 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:48.051321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:48.073878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb2edf-95fb-4576-8393-42e137e841cb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:48.148866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:48.149725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.150109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.150483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.151951 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:48.153623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.240556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1234/2405] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:48 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:48.245556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:48.255687 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736fa24-badc-46b4-9ada-687a93741aeb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:48.260354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:48.261237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.261526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.261802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.262532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:48.264357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.278162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1172/2406] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:48.279391 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:48.301777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd17e2-dc2b-47ba-866b-6b50852ff75d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:48.308415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80349de4-090b-4aa7-a042-2e6f251a8416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:48.309243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349de4-090b-4aa7-a042-2e6f251a8416 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.309750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349de4-090b-4aa7-a042-2e6f251a8416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.309998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80349de4-090b-4aa7-a042-2e6f251a8416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.310821 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80349de4-090b-4aa7-a042-2e6f251a8416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:48.312746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80349de4-090b-4aa7-a042-2e6f251a8416 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.324928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1235/2407] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:48.329046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:48.339969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df544b-96d1-4d7a-9be7-db3e427474f8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:48.345491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:48.348689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.348689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.348689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] PATH_INFO: `/v3/projects/0ea0a2c604c046f1bd3920bae4630257/users/bdba20060cb442c796a171d7c9efedfc/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.367968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea0a2c604c046f1bd3920bae4630257, user_id=bdba20060cb442c796a171d7c9efedfc, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:48.370838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.447705 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1173/2408] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:48 2022] PUT /identity/v3/projects/0ea0a2c604c046f1bd3920bae4630257/users/bdba20060cb442c796a171d7c9efedfc/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:48.457909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7d695-d15b-443f-bd95-dc95e981cd4d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.458218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7d695-d15b-443f-bd95-dc95e981cd4d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.458502 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7d695-d15b-443f-bd95-dc95e981cd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.493599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5ce7d695-d15b-443f-bd95-dc95e981cd4d None None] MFA Rules not processed for user `bdba20060cb442c796a171d7c9efedfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:48.497639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5ce7d695-d15b-443f-bd95-dc95e981cd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:48.566201 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1236/2409] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:48.571717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:48.653257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449af0e-fdd6-477e-9aa6-595c6b0f90b1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:48.660004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:48.660827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.661115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.661394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.662764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:48.665839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.759804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1174/2410] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:48 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:48.764028 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7d695-d15b-443f-bd95-dc95e981cd4d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:48.775933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7d695-d15b-443f-bd95-dc95e981cd4d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:48.858313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:48.859013 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.859298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.859575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.860296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:48.862868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.954828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1237/2411] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:48 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:48.959239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:48.970441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f3dd-564a-4bf8-9bf0-335aebe546dd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:48.975526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:48.977149 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:48.977717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:48.978003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:48.978906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:48.980552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:48.991433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1175/2412] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:21:48.996545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:49.007438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab15c3-78a2-4c90-9969-3c9dbc1c1a2a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:49.012312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:49.013208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:49.013494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:49.019020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] PATH_INFO: `/v3/projects/aad2a3f2c0254fd88e3a602df58645a9/users/fd6f2ab931f54a64b4dd8ca7ddc220aa/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:49.029476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad2a3f2c0254fd88e3a602df58645a9, user_id=fd6f2ab931f54a64b4dd8ca7ddc220aa, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:49.032252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:49.126843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1238/2413] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:48 2022] PUT /identity/v3/projects/aad2a3f2c0254fd88e3a602df58645a9/users/fd6f2ab931f54a64b4dd8ca7ddc220aa/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:49.131023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:49.141815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3d94-72e5-43c2-96da-e0c84bc0cdc7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:49.237546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315e53c-2e9f-487d-853e-ada954631694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:49.237546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e53c-2e9f-487d-853e-ada954631694 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:49.237546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e53c-2e9f-487d-853e-ada954631694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:49.237546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e53c-2e9f-487d-853e-ada954631694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:49.237899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315e53c-2e9f-487d-853e-ada954631694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:49.237899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315e53c-2e9f-487d-853e-ada954631694 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:49.260987 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1176/2414] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:21:49.264824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:49.276734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547218e-0666-47b5-9b63-51c4d3227d25 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:49.282002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:49.282942 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:49.283290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:49.283761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] PATH_INFO: `/v3/projects/aad2a3f2c0254fd88e3a602df58645a9/users/fd6f2ab931f54a64b4dd8ca7ddc220aa/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:49.285889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad2a3f2c0254fd88e3a602df58645a9, user_id=fd6f2ab931f54a64b4dd8ca7ddc220aa, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:49.289309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:49.301749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1239/2415] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:49 2022] PUT /identity/v3/projects/aad2a3f2c0254fd88e3a602df58645a9/users/fd6f2ab931f54a64b4dd8ca7ddc220aa/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:49.313850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef84c0-86f2-40f3-9216-62db1ba8fab7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:49.314395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef84c0-86f2-40f3-9216-62db1ba8fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:49.314676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef84c0-86f2-40f3-9216-62db1ba8fab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:49.359107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-89ef84c0-86f2-40f3-9216-62db1ba8fab7 None None] MFA Rules not processed for user `fd6f2ab931f54a64b4dd8ca7ddc220aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:49.360426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-89ef84c0-86f2-40f3-9216-62db1ba8fab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:49.439431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1177/2416] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:49 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:49.453718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:49.459076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:49.521592 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecf0c1-73ed-491c-b1c1-7e94412c1cd6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:49.526166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:49.527008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:49.527287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:49.529850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:49.537829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:49.539298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:49.550683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1240/2417] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:49 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:49.752119 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef84c0-86f2-40f3-9216-62db1ba8fab7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:49.761912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef84c0-86f2-40f3-9216-62db1ba8fab7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:49.765521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8c7dd-60a7-4957-b688-514ee8d263a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:49.766167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8c7dd-60a7-4957-b688-514ee8d263a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:49.766452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8c7dd-60a7-4957-b688-514ee8d263a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:49.766726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8c7dd-60a7-4957-b688-514ee8d263a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:49.826593 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8c7dd-60a7-4957-b688-514ee8d263a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:49.828075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8c7dd-60a7-4957-b688-514ee8d263a7 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:49.838326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1178/2418] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:49.990609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:49.993658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:50.124069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1462db-6cca-4c09-a0fd-013496c1e00e service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:50.130596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:50.131736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:50.132050 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:50.133102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] PATH_INFO: `/v3/users/4ad85681f27d4ae69db0deafae9ade15` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:50.139452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad85681f27d4ae69db0deafae9ade15)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:50.139452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:50.354307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad85681f27d4ae69db0deafae9ade15'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:50.434294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad85681f27d4ae69db0deafae9ade15'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:50.450246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad85681f27d4ae69db0deafae9ade15'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:50.451389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad85681f27d4ae69db0deafae9ade15'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:50.465217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad85681f27d4ae69db0deafae9ade15'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:50.471428 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1241/2419] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:49 2022] DELETE /identity/v3/users/4ad85681f27d4ae69db0deafae9ade15 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:50.477615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:50.478255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:50.478723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:50.506376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:50.508041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:50.594891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1179/2420] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:50.615252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:50.615252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:50.699006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a3b72-1504-4b27-a7b9-140010be4ad5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:50.699745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:50.700890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:50.701081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:50.701639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:50.722794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:50.724827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:50.736687 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1242/2421] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:51.417252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:51.418824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:51.431560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554b562-154a-4255-a726-c3480c2bc0c7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:51.436339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:51.442754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:51.442754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:51.442754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] PATH_INFO: `/v3/projects/53d83251ce474848bec4fe4e572698ea` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:51.442754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d83251ce474848bec4fe4e572698ea)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:51.442754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:51.567700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d83251ce474848bec4fe4e572698ea', 'request_id': 'req-965905e7-9ebc-400c-bec8-3d63e2f97dde'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:51.672519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d83251ce474848bec4fe4e572698ea', 'request_id': 'req-965905e7-9ebc-400c-bec8-3d63e2f97dde'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:51.680135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d83251ce474848bec4fe4e572698ea', 'request_id': 'req-965905e7-9ebc-400c-bec8-3d63e2f97dde'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:51.706386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] The token cache is being invalidate because project 53d83251ce474848bec4fe4e572698ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:51.707238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:51.713226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:51.715361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1180/2422] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:51 2022] DELETE /identity/v3/projects/53d83251ce474848bec4fe4e572698ea => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:51.719703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:51.736220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6df0a-177e-4fbc-9db5-8a44a2da86e8 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:51.821459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:51.822354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:51.822634 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:51.822905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] PATH_INFO: `/v3/users/6908c2423da64d459f5819c9efc59980` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:51.824364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6908c2423da64d459f5819c9efc59980)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:51.826537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:52.074305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:52.075848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:52.097223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965905e7-9ebc-400c-bec8-3d63e2f97dde admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:52.103919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:52.105171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:52.105509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:52.106009 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] PATH_INFO: `/v3/users/426aa73007cd4ca3b8c165b87f69c02e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:52.107470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=426aa73007cd4ca3b8c165b87f69c02e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:52.110356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:52.120142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6908c2423da64d459f5819c9efc59980'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.205902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6908c2423da64d459f5819c9efc59980'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.213867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6908c2423da64d459f5819c9efc59980'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.214515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6908c2423da64d459f5819c9efc59980'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.226359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6908c2423da64d459f5819c9efc59980'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.232562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1243/2423] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:51 2022] DELETE /identity/v3/users/6908c2423da64d459f5819c9efc59980 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:52.332719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426aa73007cd4ca3b8c165b87f69c02e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.419018 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426aa73007cd4ca3b8c165b87f69c02e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.419323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426aa73007cd4ca3b8c165b87f69c02e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.441648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426aa73007cd4ca3b8c165b87f69c02e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.449890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426aa73007cd4ca3b8c165b87f69c02e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.458684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1181/2424] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:52 2022] DELETE /identity/v3/users/426aa73007cd4ca3b8c165b87f69c02e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:52.746480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:52.747145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:52.750546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:52.751587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:52.853417 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9fef0-90c0-4e96-b746-d74e6f122665 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:52.857587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738b7c3-c439-4305-b064-63b3ea022fee admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:52.859698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:52.860683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:52.861612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:52.861612 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] PATH_INFO: `/v3/projects/37bccc1c6be74270967f81536d17f3d3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:52.862384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:52.863025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bccc1c6be74270967f81536d17f3d3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:52.863700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:52.864665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:52.865089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] PATH_INFO: `/v3/projects/9886c48c5650408cacaa4e7f78586404` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:52.865610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:52.867337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9886c48c5650408cacaa4e7f78586404)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:52.869924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:52.974803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9886c48c5650408cacaa4e7f78586404', 'request_id': 'req-5522eb14-a616-4057-8816-7eade1b3c9dd'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:52.999106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bccc1c6be74270967f81536d17f3d3', 'request_id': 'req-af3559e9-c690-431d-98e9-e1fbda308e72'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.004877 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bccc1c6be74270967f81536d17f3d3', 'request_id': 'req-af3559e9-c690-431d-98e9-e1fbda308e72'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.011758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bccc1c6be74270967f81536d17f3d3', 'request_id': 'req-af3559e9-c690-431d-98e9-e1fbda308e72'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.045510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9886c48c5650408cacaa4e7f78586404', 'request_id': 'req-5522eb14-a616-4057-8816-7eade1b3c9dd'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.051841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9886c48c5650408cacaa4e7f78586404', 'request_id': 'req-5522eb14-a616-4057-8816-7eade1b3c9dd'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.062861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] The token cache is being invalidate because project 9886c48c5650408cacaa4e7f78586404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:53.063144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.063823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5522eb14-a616-4057-8816-7eade1b3c9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.065374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1182/2425] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:52 2022] DELETE /identity/v3/projects/9886c48c5650408cacaa4e7f78586404 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:53.076544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf43a5-d90d-42bb-b271-b9404d27751d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.076544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf43a5-d90d-42bb-b271-b9404d27751d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.076544 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf43a5-d90d-42bb-b271-b9404d27751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.093585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1bdf43a5-d90d-42bb-b271-b9404d27751d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:53.094866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1bdf43a5-d90d-42bb-b271-b9404d27751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:53.138581 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] The token cache is being invalidate because project 37bccc1c6be74270967f81536d17f3d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:53.139165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.141434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af3559e9-c690-431d-98e9-e1fbda308e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:53.146728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1244/2426] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:52 2022] DELETE /identity/v3/projects/37bccc1c6be74270967f81536d17f3d3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:53.148780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589019f-325a-41e7-a7e4-15773d1f6eed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.149296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589019f-325a-41e7-a7e4-15773d1f6eed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.149575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589019f-325a-41e7-a7e4-15773d1f6eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.161216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1183/2427] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:53.169422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498970e-fb7b-4e6e-9f4d-24efb3f3f0ab None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.169790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498970e-fb7b-4e6e-9f4d-24efb3f3f0ab None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.170092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498970e-fb7b-4e6e-9f4d-24efb3f3f0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.174445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7589019f-325a-41e7-a7e4-15773d1f6eed None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:53.175701 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7589019f-325a-41e7-a7e4-15773d1f6eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:53.184661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1245/2428] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:53.190336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f498970e-fb7b-4e6e-9f4d-24efb3f3f0ab None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:53.191769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f498970e-fb7b-4e6e-9f4d-24efb3f3f0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:53.193717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0083c-b22f-4724-b3c7-2565101b3bc5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.194166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0083c-b22f-4724-b3c7-2565101b3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.194457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0083c-b22f-4724-b3c7-2565101b3bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.201610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1184/2429] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:53.205432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498970e-fb7b-4e6e-9f4d-24efb3f3f0ab None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.214878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498970e-fb7b-4e6e-9f4d-24efb3f3f0ab None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.216913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b7b0083c-b22f-4724-b3c7-2565101b3bc5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:53.218652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b7b0083c-b22f-4724-b3c7-2565101b3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:53.219615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.220767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.221469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.221469 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.222102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:21:53.222400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.224011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.227703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1246/2430] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:53.232044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0083c-b22f-4724-b3c7-2565101b3bc5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.235978 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1185/2431] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:21:53.240028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.241381 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0083c-b22f-4724-b3c7-2565101b3bc5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.246985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.247662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.247983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.248255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.249002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:21:53.249300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.250336 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb60ce3-5d12-4f21-ae7c-96ac3924c491 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.251074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.254985 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.255675 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.255964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.256240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.256928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.259420 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.263888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1247/2432] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:21:53.268196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.278181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364f439-c64f-4c4c-9df9-979342f352bd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.283088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.283843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.284159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.284437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.285188 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.287132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.353977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1186/2433] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:53.355069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.385460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1248/2434] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:53.386609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.423867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39883f1d-c865-403d-8794-12f229a61c2d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.466220 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62b2c-f1ba-429e-9be6-a062da7779ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.472116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.472796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.473077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.473351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.474073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.476698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.490644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.491375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.492360 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.492860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.498512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.498512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.580744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1249/2435] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:53.584435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.596293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c65d3e-0ba6-411c-9e0e-d986a7b4ea17 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.598035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1187/2436] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/users => generated 283 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:53.604886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.605066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.605560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.605838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.606108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.606828 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.608797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.619762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dec87-6c93-4615-898f-c5e79f2e5bd7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.624288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1250/2437] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:21:53.624998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.625167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.626011 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.626372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.626583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.627311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.628926 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.638910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148d806-09a0-49a3-bec4-d6512e21e7cf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.641288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1188/2438] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:53.646674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.647520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.648329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.653451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.653451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.653451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.653451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.656942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210157c-479a-4137-b602-8dcba4d56990 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.662121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.662778 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.663058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.663332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.664739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.666346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.667836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1251/2439] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:53.670504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.677597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1189/2440] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:53.681918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0b25b-d415-4955-b60e-28ced15dc25a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.686640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:53.686900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89c7d-b40d-49dc-95e0-a0777d764e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.687850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89c7d-b40d-49dc-95e0-a0777d764e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.688212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89c7d-b40d-49dc-95e0-a0777d764e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.688496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89c7d-b40d-49dc-95e0-a0777d764e32 admin admin] PATH_INFO: `/v3/projects/cddf927f7b734b41950c722f948515f3/users/e8e4290ec8c3442780ea50b2c66e4bb1/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.696588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c769-f8bc-4777-beba-f348a82bd3c4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:53.701207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c48591-c87f-425d-83d3-15845e26d213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:53.702084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c48591-c87f-425d-83d3-15845e26d213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.702363 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c48591-c87f-425d-83d3-15845e26d213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.702640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c48591-c87f-425d-83d3-15845e26d213 admin admin] PATH_INFO: `/v3/projects/e78a5a5d54134d3c937c3c0401301290/users/76d22191390e4b5e87a54839bc740934/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.704995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89c7d-b40d-49dc-95e0-a0777d764e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cddf927f7b734b41950c722f948515f3, user_id=e8e4290ec8c3442780ea50b2c66e4bb1, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.707679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89c7d-b40d-49dc-95e0-a0777d764e32 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.717204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c48591-c87f-425d-83d3-15845e26d213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78a5a5d54134d3c937c3c0401301290, user_id=76d22191390e4b5e87a54839bc740934, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:53.719838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c48591-c87f-425d-83d3-15845e26d213 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:53.797557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1252/2441] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:53 2022] PUT /identity/v3/projects/cddf927f7b734b41950c722f948515f3/users/e8e4290ec8c3442780ea50b2c66e4bb1/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:53.809352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.810151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.810441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.832958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1190/2442] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:53 2022] PUT /identity/v3/projects/e78a5a5d54134d3c937c3c0401301290/users/76d22191390e4b5e87a54839bc740934/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:53.850819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:53.852550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:53.852550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:53.858699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] MFA Rules not processed for user `e8e4290ec8c3442780ea50b2c66e4bb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:53.859939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:53.884733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] MFA Rules not processed for user `76d22191390e4b5e87a54839bc740934`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:53.885964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:53.946387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1253/2443] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:53.963339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1191/2444] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:54.074677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:54.074677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:54.080134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:54.081623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:54.133024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2babc6-53b6-4e3e-940e-e733122a9dd4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:54.136857 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:54.137648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:54.137929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:54.138028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e04e87-f49d-46f7-b312-ebad3b40e669 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:54.138200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:54.141645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5ca75-d443-4a30-abd6-1484197051b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:54.142289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5ca75-d443-4a30-abd6-1484197051b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:54.142570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5ca75-d443-4a30-abd6-1484197051b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:54.142847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5ca75-d443-4a30-abd6-1484197051b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:54.147362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:54.148866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:54.151994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5ca75-d443-4a30-abd6-1484197051b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:54.153448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5ca75-d443-4a30-abd6-1484197051b9 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:54.158056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1254/2445] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:54.162454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1192/2446] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:21:54.180790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:54.254120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770b2c5-529f-4e92-a95e-265122d44b41 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:54.258890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:54.259808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:54.260088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:54.260382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] PATH_INFO: `/v3/users/e8e4290ec8c3442780ea50b2c66e4bb1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:54.261586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e4290ec8c3442780ea50b2c66e4bb1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:54.265277 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:54.444677 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e4290ec8c3442780ea50b2c66e4bb1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:54.517372 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e4290ec8c3442780ea50b2c66e4bb1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:54.522786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e4290ec8c3442780ea50b2c66e4bb1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:54.523692 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e4290ec8c3442780ea50b2c66e4bb1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:54.536403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e4290ec8c3442780ea50b2c66e4bb1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:54.544096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1255/2447] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:54 2022] DELETE /identity/v3/users/e8e4290ec8c3442780ea50b2c66e4bb1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:54.551845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:54.551845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:54.551845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:54.567350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:54.568583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:54.628189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1193/2448] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:54.637943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:54.639401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:54.701163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0f7f2-fa57-46a7-a894-939e2ff19551 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:54.704867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:54.705506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:54.705804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:54.706059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:54.716712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:54.718165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:54.729227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1256/2449] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:55.127432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.128980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:55.144948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aba8e-fb8a-41ae-82db-ef50587156da None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.151919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.152202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.152506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.152829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] PATH_INFO: `/v3/projects/cddf927f7b734b41950c722f948515f3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.153948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cddf927f7b734b41950c722f948515f3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.155897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.208076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.209534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:55.222716 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57f594-3556-49f8-aa7d-814097e9dc72 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.228096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.229322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.229616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.229901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] PATH_INFO: `/v3/users/bdba20060cb442c796a171d7c9efedfc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.231103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdba20060cb442c796a171d7c9efedfc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.233125 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.254976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cddf927f7b734b41950c722f948515f3', 'request_id': 'req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.325952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cddf927f7b734b41950c722f948515f3', 'request_id': 'req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.332486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cddf927f7b734b41950c722f948515f3', 'request_id': 'req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.344914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] The token cache is being invalidate because project cddf927f7b734b41950c722f948515f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:55.345208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.345879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ff50d790-67d7-4481-be3b-2dbb3cc78a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.347517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1194/2450] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:55 2022] DELETE /identity/v3/projects/cddf927f7b734b41950c722f948515f3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:55.353014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb3198-1a38-4c77-8b8b-69a45a3bac8d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.353302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb3198-1a38-4c77-8b8b-69a45a3bac8d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.353581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb3198-1a38-4c77-8b8b-69a45a3bac8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.373537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3bbb3198-1a38-4c77-8b8b-69a45a3bac8d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:55.374646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3bbb3198-1a38-4c77-8b8b-69a45a3bac8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:55.433914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1195/2451] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:55.441567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34d0ab-1500-4c9d-975a-8b66e3821013 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.441974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34d0ab-1500-4c9d-975a-8b66e3821013 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.442258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34d0ab-1500-4c9d-975a-8b66e3821013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.451836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdba20060cb442c796a171d7c9efedfc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.460209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4b34d0ab-1500-4c9d-975a-8b66e3821013 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:55.462187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4b34d0ab-1500-4c9d-975a-8b66e3821013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:55.470500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1196/2452] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:55.474561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34d0ab-1500-4c9d-975a-8b66e3821013 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.484433 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34d0ab-1500-4c9d-975a-8b66e3821013 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.489058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.489680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.489966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.490240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.490957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:21:55.491302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.493528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.505102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1197/2453] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:21:55.509257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.518624 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2307-d834-4218-963a-e1351a837b4a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.526951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.527653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.527953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.528233 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.528924 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.530868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.537796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdba20060cb442c796a171d7c9efedfc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.547590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdba20060cb442c796a171d7c9efedfc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.548324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdba20060cb442c796a171d7c9efedfc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.560325 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f52b16b7-acfe-40ce-bcb7-6d16201af5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdba20060cb442c796a171d7c9efedfc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:55.567933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1257/2454] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:55 2022] DELETE /identity/v3/users/bdba20060cb442c796a171d7c9efedfc => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:55.572442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.572739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.573022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.590258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:55.591464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:55.610726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1198/2455] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:55 2022] POST /identity/v3/projects => generated 359 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:55.614970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.617056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:55.650712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1258/2456] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:55.660303 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.661994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:55.684715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76566b62-0660-47a2-add9-bd2310ffe278 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.690182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.690865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.691242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.691954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.692661 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.695311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.721524 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f394ded-13e8-435e-9cea-453352ef5470 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.774633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.775306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.775688 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.775979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.785237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.786732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.797635 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1199/2457] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:55 2022] POST /identity/v3/users => generated 280 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:55.804768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1259/2458] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:21:55.804902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.808730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca423f7-807c-4f92-8b94-a6666facd997 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.813597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.814438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.814721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.815001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.815836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.817706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.828831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1200/2459] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:55.832950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.846789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc1eed-c9c5-430d-8c6d-3bd6fef6abf7 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.852593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3d125-3f1d-4b5d-b626-241aad8757fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.853262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3d125-3f1d-4b5d-b626-241aad8757fe admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.853545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3d125-3f1d-4b5d-b626-241aad8757fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.853822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3d125-3f1d-4b5d-b626-241aad8757fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.854744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3d125-3f1d-4b5d-b626-241aad8757fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.856218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3d125-3f1d-4b5d-b626-241aad8757fe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:55.866349 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1260/2460] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:55.870885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:55.881288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87196-64c8-4a75-8f50-70cb403be183 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:55.887057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:55.888203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:55.888552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:55.888902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] PATH_INFO: `/v3/projects/f0ffe0c3506340448a9422add7254737/users/054dec6296ce40f9aeb87d4e341a63a5/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:55.905727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ffe0c3506340448a9422add7254737, user_id=054dec6296ce40f9aeb87d4e341a63a5, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:55.908476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:56.052630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1201/2461] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:55 2022] PUT /identity/v3/projects/f0ffe0c3506340448a9422add7254737/users/054dec6296ce40f9aeb87d4e341a63a5/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:56.066874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.067387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.067945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.099299 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] MFA Rules not processed for user `054dec6296ce40f9aeb87d4e341a63a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:56.100623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.177264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1261/2462] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:56 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:56.208137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:56.209911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.264907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb6dd2-c2f1-4906-bb94-8e1644d80d15 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:56.266948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:56.268468 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.272158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:56.272158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.272158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.272158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.281382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:56.283010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:56.299779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1202/2463] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:56.316551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:56.345337 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee862-96ed-4e6c-9e89-c53c2de5b110 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:56.349949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:56.350801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.351886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.352169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] PATH_INFO: `/v3/projects/0ea0a2c604c046f1bd3920bae4630257` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.353364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea0a2c604c046f1bd3920bae4630257)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:56.355270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:56.382464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3192a8f-1f08-447e-b5ee-91b4ca5d6713 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:56.387521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:56.388437 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.388717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.388999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] PATH_INFO: `/v3/users/054dec6296ce40f9aeb87d4e341a63a5` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.390226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054dec6296ce40f9aeb87d4e341a63a5)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:56.392333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:56.488327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea0a2c604c046f1bd3920bae4630257', 'request_id': 'req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.493651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea0a2c604c046f1bd3920bae4630257', 'request_id': 'req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.499595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea0a2c604c046f1bd3920bae4630257', 'request_id': 'req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.565423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] The token cache is being invalidate because project 0ea0a2c604c046f1bd3920bae4630257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:56.565709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.566462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.568106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1262/2464] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:56 2022] DELETE /identity/v3/projects/0ea0a2c604c046f1bd3920bae4630257 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:56.572585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:56.573846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.603712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054dec6296ce40f9aeb87d4e341a63a5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.658507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8d548-81dd-4e81-8f2a-43454ba0c34b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:56.661907 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:56.663058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.663475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.664077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] PATH_INFO: `/v3/users/fd6f2ab931f54a64b4dd8ca7ddc220aa` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.666002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6f2ab931f54a64b4dd8ca7ddc220aa)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:56.668283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:56.687175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054dec6296ce40f9aeb87d4e341a63a5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.688124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054dec6296ce40f9aeb87d4e341a63a5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.708101 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054dec6296ce40f9aeb87d4e341a63a5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.718962 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054dec6296ce40f9aeb87d4e341a63a5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.733452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1203/2465] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:56 2022] DELETE /identity/v3/users/054dec6296ce40f9aeb87d4e341a63a5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:56.734499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:56.736260 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.800073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119de01a-7d37-4c78-abd3-3e9e54210084 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:56.803717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100c8ef-ab18-4ae4-aff5-a66c620252c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:56.804426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100c8ef-ab18-4ae4-aff5-a66c620252c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.804873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100c8ef-ab18-4ae4-aff5-a66c620252c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.805153 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100c8ef-ab18-4ae4-aff5-a66c620252c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.806999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b100c8ef-ab18-4ae4-aff5-a66c620252c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.862516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6f2ab931f54a64b4dd8ca7ddc220aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.872299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100c8ef-ab18-4ae4-aff5-a66c620252c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:56.873750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100c8ef-ab18-4ae4-aff5-a66c620252c7 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:56.884818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1204/2466] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:21:56.886300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.886591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.886870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.906293 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:56.907610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.916388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1205/2467] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:56.926879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:56.928640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.930297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6f2ab931f54a64b4dd8ca7ddc220aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.938556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6f2ab931f54a64b4dd8ca7ddc220aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.939254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6f2ab931f54a64b4dd8ca7ddc220aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.941961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e00ee-b09b-4768-a0d0-0985493c0621 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:56.947747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:56.949344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:56.949781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:56.951728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:56.953083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6f2ab931f54a64b4dd8ca7ddc220aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:56.953737 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:56.959932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1263/2468] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:56 2022] DELETE /identity/v3/users/fd6f2ab931f54a64b4dd8ca7ddc220aa => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:56.970388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:56.971959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:56.982324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1206/2469] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:21:57.472912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:57.479187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:57.548173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e92346-afa6-4dbd-8f6c-9a7edb7fddd7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:57.553081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:57.554044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:57.554511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:57.554856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] PATH_INFO: `/v3/projects/aad2a3f2c0254fd88e3a602df58645a9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:57.556174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad2a3f2c0254fd88e3a602df58645a9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:57.558238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:57.617822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:57.619314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:57.631107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f663-64f4-429d-b90a-e9aa4d0d0b1f service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:57.636618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:57.639775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:57.639775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:57.639775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] PATH_INFO: `/v3/projects/f0ffe0c3506340448a9422add7254737` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:57.639775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ffe0c3506340448a9422add7254737)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:57.641242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:57.682437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad2a3f2c0254fd88e3a602df58645a9', 'request_id': 'req-756779f8-f0f7-439c-a2be-811fe6d9f564'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.687962 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad2a3f2c0254fd88e3a602df58645a9', 'request_id': 'req-756779f8-f0f7-439c-a2be-811fe6d9f564'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.694275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad2a3f2c0254fd88e3a602df58645a9', 'request_id': 'req-756779f8-f0f7-439c-a2be-811fe6d9f564'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.770377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ffe0c3506340448a9422add7254737', 'request_id': 'req-7cb7e771-2647-4ed3-bbac-74827256ae0e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.785509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] The token cache is being invalidate because project aad2a3f2c0254fd88e3a602df58645a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:57.785930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.786685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-756779f8-f0f7-439c-a2be-811fe6d9f564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.789504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1264/2470] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:57 2022] DELETE /identity/v3/projects/aad2a3f2c0254fd88e3a602df58645a9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:57.794166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49b76e-a138-4248-a466-8117991a5b1b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:57.794450 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49b76e-a138-4248-a466-8117991a5b1b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:57.794725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49b76e-a138-4248-a466-8117991a5b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:57.815208 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5c49b76e-a138-4248-a466-8117991a5b1b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:57.816437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c49b76e-a138-4248-a466-8117991a5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:57.857145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ffe0c3506340448a9422add7254737', 'request_id': 'req-7cb7e771-2647-4ed3-bbac-74827256ae0e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.863403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ffe0c3506340448a9422add7254737', 'request_id': 'req-7cb7e771-2647-4ed3-bbac-74827256ae0e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.877424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1265/2471] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:57.886206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566ffb2-a50f-4095-8f86-a2f55c0d7e48 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:57.886652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566ffb2-a50f-4095-8f86-a2f55c0d7e48 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:57.887014 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566ffb2-a50f-4095-8f86-a2f55c0d7e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:57.889006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] The token cache is being invalidate because project f0ffe0c3506340448a9422add7254737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:21:57.889315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.889956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7cb7e771-2647-4ed3-bbac-74827256ae0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:57.892208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1207/2472] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:21:57 2022] DELETE /identity/v3/projects/f0ffe0c3506340448a9422add7254737 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:57.897121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229f016-7ea7-448f-a9d1-a2474dc68ac8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:57.897477 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229f016-7ea7-448f-a9d1-a2474dc68ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:57.897867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229f016-7ea7-448f-a9d1-a2474dc68ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:57.908655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0566ffb2-a50f-4095-8f86-a2f55c0d7e48 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:57.910057 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0566ffb2-a50f-4095-8f86-a2f55c0d7e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:57.917983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8229f016-7ea7-448f-a9d1-a2474dc68ac8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:57.919403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8229f016-7ea7-448f-a9d1-a2474dc68ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:57.933444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1266/2473] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:57.941401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1208/2474] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:57.943567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978db55d-6ca4-4475-bd4f-5c5fbc5536f0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:57.944704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978db55d-6ca4-4475-bd4f-5c5fbc5536f0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:57.945818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978db55d-6ca4-4475-bd4f-5c5fbc5536f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:57.947993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229f016-7ea7-448f-a9d1-a2474dc68ac8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:57.963893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229f016-7ea7-448f-a9d1-a2474dc68ac8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:57.967787 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-978db55d-6ca4-4475-bd4f-5c5fbc5536f0 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:57.969211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-978db55d-6ca4-4475-bd4f-5c5fbc5536f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:57.969332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:57.970090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:57.970564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:57.970905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:57.971806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:21:57.972221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:57.973907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:57.981762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1267/2475] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:57.985437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978db55d-6ca4-4475-bd4f-5c5fbc5536f0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:57.992508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1209/2476] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:21:57.997117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.001623 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978db55d-6ca4-4475-bd4f-5c5fbc5536f0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.008265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.009399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.010003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.010364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417529-b1fb-400b-a5fa-c9965b414f8d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.010747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.011911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:21:58.012715 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.015029 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.015529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.020008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.020514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.021014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.031603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.033612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.036396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1268/2477] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:21:58.039991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.053087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd6af-0458-42b7-9c8b-43ee78c2fbeb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.062341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.063792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.063792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.063971 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.064776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.066908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.182710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1210/2478] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:57 2022] POST /identity/v3/projects => generated 367 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:58.186449 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.187296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1269/2479] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:58 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:58.191165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.258128 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51940ae4-94e6-4914-aec8-f1737441f4a1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.258519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600cb58-8ad5-4cec-8776-52004d492172 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.263126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.263970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.264112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.264267 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.264548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.265238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.265370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.265519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.265798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.266492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.267211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.268427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.352550 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1270/2480] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:58 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:58.353986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.364497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10191d08-7136-460b-9269-739258e48b40 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.369499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.371947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.371947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.371947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.371947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.373129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.379994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1211/2481] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:58 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:58.385746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.391516 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1271/2482] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:21:58.392436 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.407811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fae81-e7f3-4fb2-8a8d-1c5c0c90b90e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.416481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.417927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.421880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.421880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.421880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.421880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.423338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5322-3b9a-4258-b00b-71dd5a1f4493 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.429006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.429786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.430079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.430471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.431781 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.434157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.437391 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1272/2483] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:58.441613 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.450767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1212/2484] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:58.452113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.452732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc9fe-f661-4e84-8c70-6d548bc1e89f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.457753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4661fc-19aa-49bf-9100-12a7734601eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.459059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4661fc-19aa-49bf-9100-12a7734601eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.459479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4661fc-19aa-49bf-9100-12a7734601eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.459796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4661fc-19aa-49bf-9100-12a7734601eb admin admin] PATH_INFO: `/v3/projects/ca801249ec5441a7b977ffc4438e3b87/users/e3fa974a0f7b4eaf8b333b9828d98a35/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.473739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715a9c3-eac9-4d6e-924f-60918cf09071 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.484786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.485526 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.486024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.486310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.487676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.489793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.490048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4661fc-19aa-49bf-9100-12a7734601eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca801249ec5441a7b977ffc4438e3b87, user_id=e3fa974a0f7b4eaf8b333b9828d98a35, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.493820 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4661fc-19aa-49bf-9100-12a7734601eb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.503364 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1213/2485] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:58.508045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.520077 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a06903-7f0b-4ee3-a216-90e4d9257b8e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.526516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca244f6-ff3e-4f25-b32d-85a79d42190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.527859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca244f6-ff3e-4f25-b32d-85a79d42190d admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.527981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca244f6-ff3e-4f25-b32d-85a79d42190d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.528374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca244f6-ff3e-4f25-b32d-85a79d42190d admin admin] PATH_INFO: `/v3/projects/46876a2b970f44cf871a1ff42ee6a06c/users/5d59e244dceb4d25a4bbe888c9a42002/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.547756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca244f6-ff3e-4f25-b32d-85a79d42190d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46876a2b970f44cf871a1ff42ee6a06c, user_id=5d59e244dceb4d25a4bbe888c9a42002, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.551386 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca244f6-ff3e-4f25-b32d-85a79d42190d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.583994 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1273/2486] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:58 2022] PUT /identity/v3/projects/ca801249ec5441a7b977ffc4438e3b87/users/e3fa974a0f7b4eaf8b333b9828d98a35/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:58.596046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.596494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.596820 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.627015 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] MFA Rules not processed for user `e3fa974a0f7b4eaf8b333b9828d98a35`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:58.630310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:58.635447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1214/2487] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:58 2022] PUT /identity/v3/projects/46876a2b970f44cf871a1ff42ee6a06c/users/5d59e244dceb4d25a4bbe888c9a42002/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:58.659041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792de1-9556-4fcc-b49f-f1a14bfb010d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.659808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792de1-9556-4fcc-b49f-f1a14bfb010d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.660326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792de1-9556-4fcc-b49f-f1a14bfb010d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.702875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-fb792de1-9556-4fcc-b49f-f1a14bfb010d None None] MFA Rules not processed for user `5d59e244dceb4d25a4bbe888c9a42002`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:58.704496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fb792de1-9556-4fcc-b49f-f1a14bfb010d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:58.733635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1274/2488] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:58.746845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.748643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:58.790273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1215/2489] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:58.795271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792de1-9556-4fcc-b49f-f1a14bfb010d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.813861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fe8fa-98ff-4efc-8757-34b9db5820df None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.818182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.818841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.819127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.819564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.830307 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.831495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.843086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1275/2490] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:21:58.858830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792de1-9556-4fcc-b49f-f1a14bfb010d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.864021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.864655 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.864967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.865202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.865976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.868004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.872551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:58.886873 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28e3ef-621c-41a4-8114-8344ee510507 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:58.892109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:58.895840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:58.895840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:58.895840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] PATH_INFO: `/v3/users/e3fa974a0f7b4eaf8b333b9828d98a35` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:58.895840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3fa974a0f7b4eaf8b333b9828d98a35)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:58.897362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:58.987523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1216/2491] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:21:58 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:21:58.991402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:59.062769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b2bf-e0cb-414f-8acf-eda086e5bb08 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:59.067531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:59.068241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.068527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.068812 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.069537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:59.072554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:59.093072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3fa974a0f7b4eaf8b333b9828d98a35'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:59.163806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1217/2492] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:21:58 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:21:59.167682 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:59.178323 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a008c6b-d01b-475f-84d0-ea57bcbe69c7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:59.180704 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3fa974a0f7b4eaf8b333b9828d98a35'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:59.184172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:59.184976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.185321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.185601 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.186357 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:59.187968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:59.191732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3fa974a0f7b4eaf8b333b9828d98a35'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:59.191732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3fa974a0f7b4eaf8b333b9828d98a35'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:59.199176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1218/2493] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:59.203217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:59.204907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:59.208973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f59e18d1-e649-4fd7-abf8-cd3c134108d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3fa974a0f7b4eaf8b333b9828d98a35'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:21:59.216293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1276/2494] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:21:58 2022] DELETE /identity/v3/users/e3fa974a0f7b4eaf8b333b9828d98a35 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:21:59.220434 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76792c-a9f5-46c8-9b26-2789519bd353 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:59.223862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.224222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.224730 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.242838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:59.246074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:59.278770 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:59.279835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.280115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.280384 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] PATH_INFO: `/v3/projects/a8a8080a0e1d43d1a87f6aec46723372/users/3cdba89fe20c4e4a914b81e6aaa72691/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.298251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a8080a0e1d43d1a87f6aec46723372, user_id=3cdba89fe20c4e4a914b81e6aaa72691, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:59.301234 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:59.309044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1277/2495] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:21:59.320280 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:59.321754 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:59.380675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919276b-cdeb-4a13-ae7e-5515116601df None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:59.386228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1219/2496] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:59 2022] PUT /identity/v3/projects/a8a8080a0e1d43d1a87f6aec46723372/users/3cdba89fe20c4e4a914b81e6aaa72691/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:59.387542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:59.441724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f863072-97a7-4459-b5bb-5014b7d7d4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:59.443680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f863072-97a7-4459-b5bb-5014b7d7d4fb service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.443680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f863072-97a7-4459-b5bb-5014b7d7d4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.443680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f863072-97a7-4459-b5bb-5014b7d7d4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.477250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e81d-9f03-44d7-b4e6-795f33dafe71 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:59.481775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:59.482542 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.482901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.483432 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.484288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:59.486549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:59.502950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1220/2497] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:21:59.504081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:59.519003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780541a8-3442-4d92-8b92-457339a06094 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:59.525219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f863072-97a7-4459-b5bb-5014b7d7d4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:59.526100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:59.526791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f863072-97a7-4459-b5bb-5014b7d7d4fb service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:59.527499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.527918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.528314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] PATH_INFO: `/v3/projects/a8a8080a0e1d43d1a87f6aec46723372/users/3cdba89fe20c4e4a914b81e6aaa72691/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.530568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a8080a0e1d43d1a87f6aec46723372, user_id=3cdba89fe20c4e4a914b81e6aaa72691, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:59.533483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:59.539880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1278/2498] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:21:59.564284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1221/2499] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:21:59 2022] PUT /identity/v3/projects/a8a8080a0e1d43d1a87f6aec46723372/users/3cdba89fe20c4e4a914b81e6aaa72691/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:21:59.569373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485ff3b-a5f2-423e-a09a-d8ceda22a3f5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.573486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485ff3b-a5f2-423e-a09a-d8ceda22a3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.573486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485ff3b-a5f2-423e-a09a-d8ceda22a3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.615161 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1485ff3b-a5f2-423e-a09a-d8ceda22a3f5 None None] MFA Rules not processed for user `3cdba89fe20c4e4a914b81e6aaa72691`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:21:59.617251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1485ff3b-a5f2-423e-a09a-d8ceda22a3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:59.702205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1279/2500] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:21:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:21:59.713559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:21:59.718104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:21:59.780510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a13d0-60c3-4ce1-90c5-493fa3c2225e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:21:59.784437 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:21:59.786837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:21:59.786837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:21:59.786837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:21:59.796763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:21:59.798332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:21:59.819067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1222/2501] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:21:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:00.097911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485ff3b-a5f2-423e-a09a-d8ceda22a3f5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:00.112368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:00.113774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485ff3b-a5f2-423e-a09a-d8ceda22a3f5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:00.113915 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:00.119388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd60e4-26bb-481b-b159-21f34bd715d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:00.123608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd60e4-26bb-481b-b159-21f34bd715d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:00.123608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd60e4-26bb-481b-b159-21f34bd715d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:00.123608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd60e4-26bb-481b-b159-21f34bd715d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:00.123608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bfbd60e4-26bb-481b-b159-21f34bd715d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:00.220746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd60e4-26bb-481b-b159-21f34bd715d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:00.222497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd60e4-26bb-481b-b159-21f34bd715d4 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:00.238892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1280/2502] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:00.249988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e723180-c3bc-4d87-8de6-c05368cbb152 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:00.255793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:00.256724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:00.257013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:00.257326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] PATH_INFO: `/v3/projects/ca801249ec5441a7b977ffc4438e3b87` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:00.258835 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca801249ec5441a7b977ffc4438e3b87)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:00.260825 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:00.428905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca801249ec5441a7b977ffc4438e3b87', 'request_id': 'req-68907a6f-24e5-42ef-80e8-9f98880ca692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:00.492110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca801249ec5441a7b977ffc4438e3b87', 'request_id': 'req-68907a6f-24e5-42ef-80e8-9f98880ca692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:00.499295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca801249ec5441a7b977ffc4438e3b87', 'request_id': 'req-68907a6f-24e5-42ef-80e8-9f98880ca692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:00.528816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] The token cache is being invalidate because project ca801249ec5441a7b977ffc4438e3b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:00.529345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:00.530067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-68907a6f-24e5-42ef-80e8-9f98880ca692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:00.532539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1223/2503] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:00 2022] DELETE /identity/v3/projects/ca801249ec5441a7b977ffc4438e3b87 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:00.537203 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f376ce6-8a81-4729-9dee-8b26ad481c2b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:00.537685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f376ce6-8a81-4729-9dee-8b26ad481c2b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:00.537808 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f376ce6-8a81-4729-9dee-8b26ad481c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:00.563595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2f376ce6-8a81-4729-9dee-8b26ad481c2b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:00.566332 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2f376ce6-8a81-4729-9dee-8b26ad481c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:00.656783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1281/2504] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:00.725597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea904-3c88-403e-8de9-eaa45023b0f6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:00.726760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea904-3c88-403e-8de9-eaa45023b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:00.727115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea904-3c88-403e-8de9-eaa45023b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:00.745560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ba7ea904-3c88-403e-8de9-eaa45023b0f6 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:00.746776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ba7ea904-3c88-403e-8de9-eaa45023b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:00.754666 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1224/2505] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:00.758960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f376ce6-8a81-4729-9dee-8b26ad481c2b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:00.769641 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f376ce6-8a81-4729-9dee-8b26ad481c2b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:00.775978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:00.777087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:00.777401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:00.777723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:00.778465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:00.778778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:00.780424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:00.800587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1282/2506] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:22:00.807390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea904-3c88-403e-8de9-eaa45023b0f6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:00.816879 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea904-3c88-403e-8de9-eaa45023b0f6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:00.826343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:00.826343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:00.826343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:00.826343 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:00.826732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:00.826732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:00.978251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1225/2507] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:00 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:00.982265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:00.982696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:00.984855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:00.999410 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b1e10-c2ac-447e-ad03-6108a3e33d3b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:01.066600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:01.067735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.068018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.068290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] PATH_INFO: `/v3/users/7b559cc982a846bea7f7f2077b7c455b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.069389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b082b-741a-47e3-b57a-44b34c4af2ff admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:01.069540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b559cc982a846bea7f7f2077b7c455b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:01.072027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:01.074905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:01.075966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.076242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.076438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.077164 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:01.079161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:01.183711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1226/2508] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:00 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:01.184870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:01.196147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a391c5-bbc9-469d-a3ea-5a1a42cba0b1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:01.200849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:01.201581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.201863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.202141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.202884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:01.204690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:01.220486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1227/2509] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:01.221447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:01.231808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60beb9-21cd-4c22-a10c-259607cadf1d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:01.239476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:01.242300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.242300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.242515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.243870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:01.246918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:01.258630 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1228/2510] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:01.262720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:01.271872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b559cc982a846bea7f7f2077b7c455b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.273734 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8baaba-1141-4453-9420-09373b7d85de admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:01.278919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:01.281846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.281846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.281846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] PATH_INFO: `/v3/projects/8141ea82ed35431693530da3cf074b46/users/d0154612674d46c8ae3f75ff3aef4a00/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.295700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8141ea82ed35431693530da3cf074b46, user_id=d0154612674d46c8ae3f75ff3aef4a00, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:01.298377 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:01.340096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b559cc982a846bea7f7f2077b7c455b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.347718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b559cc982a846bea7f7f2077b7c455b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.350724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b559cc982a846bea7f7f2077b7c455b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.360601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c48a084-ba79-4774-b3c8-9b2cf8a7a8df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b559cc982a846bea7f7f2077b7c455b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.366596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1283/2511] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:00 2022] DELETE /identity/v3/users/7b559cc982a846bea7f7f2077b7c455b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:01.391380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1229/2512] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:01 2022] PUT /identity/v3/projects/8141ea82ed35431693530da3cf074b46/users/d0154612674d46c8ae3f75ff3aef4a00/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:01.402939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.403266 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.403801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.432370 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] MFA Rules not processed for user `d0154612674d46c8ae3f75ff3aef4a00`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:01.433729 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:01.499005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1284/2513] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:01.510613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:01.512727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:01.570397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092db2f-04d9-4c85-98ea-2b75587f2950 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:01.574193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa741473-b474-4617-b361-38c400639ae3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:01.574841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa741473-b474-4617-b361-38c400639ae3 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.575130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa741473-b474-4617-b361-38c400639ae3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.575407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa741473-b474-4617-b361-38c400639ae3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.583660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa741473-b474-4617-b361-38c400639ae3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:01.585084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa741473-b474-4617-b361-38c400639ae3 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:01.594243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1230/2514] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:01.637187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:01.638278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:01.714221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dd3f9-84da-48ed-8158-b8159288f38d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:01.718881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:01.724355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.724355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.724355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] PATH_INFO: `/v3/projects/64990eb359b54b0a8636a6e7619800a7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.724355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64990eb359b54b0a8636a6e7619800a7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:01.724355 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:01.828773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64990eb359b54b0a8636a6e7619800a7', 'request_id': 'req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.833876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64990eb359b54b0a8636a6e7619800a7', 'request_id': 'req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.840589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64990eb359b54b0a8636a6e7619800a7', 'request_id': 'req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.939534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] The token cache is being invalidate because project 64990eb359b54b0a8636a6e7619800a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:01.939859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.940478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8305d8b5-6862-4c96-8d77-ce37c9d98f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:01.942094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1285/2515] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:01 2022] DELETE /identity/v3/projects/64990eb359b54b0a8636a6e7619800a7 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:01.947767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb1af5-2886-4f93-a83c-40ea63a7f89f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:01.948248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb1af5-2886-4f93-a83c-40ea63a7f89f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:01.948424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb1af5-2886-4f93-a83c-40ea63a7f89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:01.967068 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d5cb1af5-2886-4f93-a83c-40ea63a7f89f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:01.968976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d5cb1af5-2886-4f93-a83c-40ea63a7f89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:02.027576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1231/2516] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:02.035202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32a657-0680-469c-9179-aae0213221ca None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.035490 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32a657-0680-469c-9179-aae0213221ca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.035860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32a657-0680-469c-9179-aae0213221ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.059268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5e32a657-0680-469c-9179-aae0213221ca None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:02.060480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e32a657-0680-469c-9179-aae0213221ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:02.073076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1286/2517] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:02.076033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb1af5-2886-4f93-a83c-40ea63a7f89f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.088158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb1af5-2886-4f93-a83c-40ea63a7f89f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.093112 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.094104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.094399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.094682 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.095455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:02.095852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.103967 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:02.117238 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1232/2518] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:22:02.121662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32a657-0680-469c-9179-aae0213221ca None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.133542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32a657-0680-469c-9179-aae0213221ca None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.138481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.139171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.139454 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.139792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.140487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.142434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:02.250796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1287/2519] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:02 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:02.251892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.262968 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7677-10e2-4063-bb98-30d361bfac72 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.327822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.328501 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.328782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.329056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.329753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.331911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:02.423298 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1233/2520] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:02 2022] POST /identity/v3/users => generated 303 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:02.424909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.435977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc56c6-8244-4192-827f-8c2a9f5b90a7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.441847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.442533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.442836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.443129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.443903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.446229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:02.456937 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1288/2521] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:02.460965 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.470828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd1763-1642-4250-a9b1-78e2013a113c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.475550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f6168-354e-49bd-b1c7-e4c64c582e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.476839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f6168-354e-49bd-b1c7-e4c64c582e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.477118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f6168-354e-49bd-b1c7-e4c64c582e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.477390 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f6168-354e-49bd-b1c7-e4c64c582e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.478150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f6168-354e-49bd-b1c7-e4c64c582e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.480190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f6168-354e-49bd-b1c7-e4c64c582e69 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:02.490464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1234/2522] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:02.495395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.505424 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6609e7-906e-4a78-9f21-6b6d920c00df admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.510112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.512815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.512815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.512815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] PATH_INFO: `/v3/projects/d446806bb52d4e10966cc52d5c0f2468/users/f7bfe51c415b4b28984ed00c30a0728d/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.525898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d446806bb52d4e10966cc52d5c0f2468, user_id=f7bfe51c415b4b28984ed00c30a0728d, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.528606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:02.600293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1289/2523] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:02 2022] PUT /identity/v3/projects/d446806bb52d4e10966cc52d5c0f2468/users/f7bfe51c415b4b28984ed00c30a0728d/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:02.611930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dbb73-18f6-4449-b214-e5d5cd897a74 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.612220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dbb73-18f6-4449-b214-e5d5cd897a74 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.612496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dbb73-18f6-4449-b214-e5d5cd897a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.639271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7c5dbb73-18f6-4449-b214-e5d5cd897a74 None None] MFA Rules not processed for user `f7bfe51c415b4b28984ed00c30a0728d`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:02.640508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7c5dbb73-18f6-4449-b214-e5d5cd897a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:02.701241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1235/2524] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:02.705651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.766086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e6b13-3ba9-42ec-952d-762dacf22884 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.770973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.771919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.772202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.772476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.773315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.775234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:02.867916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1290/2525] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:02 2022] POST /identity/v3/projects => generated 405 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:02.872032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dbb73-18f6-4449-b214-e5d5cd897a74 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:02.884025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dbb73-18f6-4449-b214-e5d5cd897a74 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:02.939558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:02.940267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:02.940577 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:02.940849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:02.941537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:02.943431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:03.058641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1236/2526] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:02 2022] POST /identity/v3/users => generated 309 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:03.064374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:03.074888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366558-dfc1-4a1a-9798-e8452f4717cf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:03.079507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:03.080186 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.080465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.080734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.081442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:03.083030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:03.094946 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1291/2527] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:03.099127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:03.109478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42272d-1b6a-48cb-871b-cc5ac48d3741 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:03.114072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2979d6f-ea4e-4b9d-9d29-336f4a994994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:03.114947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979d6f-ea4e-4b9d-9d29-336f4a994994 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.115228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979d6f-ea4e-4b9d-9d29-336f4a994994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.115505 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979d6f-ea4e-4b9d-9d29-336f4a994994 admin admin] PATH_INFO: `/v3/projects/9e1e4329a8b2409ab4e4bfa0ce4e0323/users/1ab9dd8fb53c417a8e8ef031d132afae/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.130718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2979d6f-ea4e-4b9d-9d29-336f4a994994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1e4329a8b2409ab4e4bfa0ce4e0323, user_id=1ab9dd8fb53c417a8e8ef031d132afae, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:03.133693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2979d6f-ea4e-4b9d-9d29-336f4a994994 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:03.232208 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1237/2528] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:03 2022] PUT /identity/v3/projects/9e1e4329a8b2409ab4e4bfa0ce4e0323/users/1ab9dd8fb53c417a8e8ef031d132afae/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:03.236350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:03.246568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912d44f-75f4-4d64-8d6c-39a1fc286d05 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:03.289618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.289944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.290224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.307193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] MFA Rules not processed for user `d0154612674d46c8ae3f75ff3aef4a00`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:03.308540 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:03.315017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:03.315017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.315017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.315017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.315555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:03.315555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:03.327848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1292/2529] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:03.329062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:03.338973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916905bb-bd8b-412b-a5b3-23b856722e9e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:03.343486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c35f9e-145a-47b9-95e3-1d843e86fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:03.346501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c35f9e-145a-47b9-95e3-1d843e86fc6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.346501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c35f9e-145a-47b9-95e3-1d843e86fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.346501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c35f9e-145a-47b9-95e3-1d843e86fc6f admin admin] PATH_INFO: `/v3/projects/9e1e4329a8b2409ab4e4bfa0ce4e0323/users/1ab9dd8fb53c417a8e8ef031d132afae/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.346916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c35f9e-145a-47b9-95e3-1d843e86fc6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1e4329a8b2409ab4e4bfa0ce4e0323, user_id=1ab9dd8fb53c417a8e8ef031d132afae, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:03.349537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c35f9e-145a-47b9-95e3-1d843e86fc6f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:03.363894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1293/2530] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:03 2022] PUT /identity/v3/projects/9e1e4329a8b2409ab4e4bfa0ce4e0323/users/1ab9dd8fb53c417a8e8ef031d132afae/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:03.369057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1238/2531] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:03.370832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b430c3-dafd-45fc-9043-77cff0b0dd7f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.371308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b430c3-dafd-45fc-9043-77cff0b0dd7f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.371791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b430c3-dafd-45fc-9043-77cff0b0dd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.402988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-53b430c3-dafd-45fc-9043-77cff0b0dd7f None None] MFA Rules not processed for user `1ab9dd8fb53c417a8e8ef031d132afae`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:03.404264 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-53b430c3-dafd-45fc-9043-77cff0b0dd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:03.488069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1294/2532] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:03.499517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:03.501468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:03.559247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013984a-7251-4bf8-95f7-0d0a2bed56ec None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:03.607743 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef307c-5546-41cc-9ab5-c09ee2dfd9b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:03.608316 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef307c-5546-41cc-9ab5-c09ee2dfd9b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.608594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef307c-5546-41cc-9ab5-c09ee2dfd9b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.608865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef307c-5546-41cc-9ab5-c09ee2dfd9b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.618428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef307c-5546-41cc-9ab5-c09ee2dfd9b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:03.619882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef307c-5546-41cc-9ab5-c09ee2dfd9b1 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:03.630138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1239/2533] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:03 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:03.785180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b430c3-dafd-45fc-9043-77cff0b0dd7f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:03.847012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b430c3-dafd-45fc-9043-77cff0b0dd7f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:03.852169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:03.853082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:03.853404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:03.853700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] PATH_INFO: `/v3/users/f7bfe51c415b4b28984ed00c30a0728d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:03.854880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bfe51c415b4b28984ed00c30a0728d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:03.856888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:04.072811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bfe51c415b4b28984ed00c30a0728d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.128421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bfe51c415b4b28984ed00c30a0728d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.136757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bfe51c415b4b28984ed00c30a0728d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.137713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bfe51c415b4b28984ed00c30a0728d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.151290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bfe51c415b4b28984ed00c30a0728d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.157254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1295/2534] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:03 2022] DELETE /identity/v3/users/f7bfe51c415b4b28984ed00c30a0728d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:04.166175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:04.166175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:04.166175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:04.182828 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:04.186160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:04.244050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1240/2535] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:04.253745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:04.255279 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:04.314427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9f504-76db-45bf-a9b3-eb851f14bdb2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:04.318608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:04.320262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:04.320262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:04.324321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:04.329986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:04.331412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:04.341451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1296/2536] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:04.463321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:04.464827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:04.477057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27851e11-7a69-4c36-ba97-fc516765cbba None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:04.481752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:04.484937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:04.484937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:04.484937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] PATH_INFO: `/v3/users/76d22191390e4b5e87a54839bc740934` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:04.484937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d22191390e4b5e87a54839bc740934)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:04.486301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:04.750921 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d22191390e4b5e87a54839bc740934'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.813522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:04.814950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:04.828314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82965-cae2-40e8-9b36-e922bd011a7c service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:04.833463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d22191390e4b5e87a54839bc740934'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.834457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d22191390e4b5e87a54839bc740934'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.834836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:04.836955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:04.837461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:04.843403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] PATH_INFO: `/v3/projects/d446806bb52d4e10966cc52d5c0f2468` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:04.843403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d446806bb52d4e10966cc52d5c0f2468)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:04.843403 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:04.859295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d22191390e4b5e87a54839bc740934'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.868998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d05f1463-833e-4810-9a9a-cab038e20735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d22191390e4b5e87a54839bc740934'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.879012 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1241/2537] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:04 2022] DELETE /identity/v3/users/76d22191390e4b5e87a54839bc740934 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:04.884038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:04.884398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:04.884741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:04.906857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:04.906857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:04.962184 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1242/2538] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:04.971840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:04.973182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:04.980350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd446806bb52d4e10966cc52d5c0f2468', 'request_id': 'req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.985823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd446806bb52d4e10966cc52d5c0f2468', 'request_id': 'req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:04.991918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd446806bb52d4e10966cc52d5c0f2468', 'request_id': 'req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.030576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34adfac-60ba-4501-bfe2-e1f114b4fc79 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:05.083321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:05.083321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:05.083321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:05.083321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:05.126421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] The token cache is being invalidate because project d446806bb52d4e10966cc52d5c0f2468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:05.126924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.127620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.129646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1297/2539] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:04 2022] DELETE /identity/v3/projects/d446806bb52d4e10966cc52d5c0f2468 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:05.134293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:05.135876 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:05.154908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:05.156900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:05.166883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1243/2540] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:05.205476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc84bbe-d820-4ac6-af2c-21e17b54d8a3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:05.210234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:05.211112 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:05.211382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:05.211672 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] PATH_INFO: `/v3/users/1ab9dd8fb53c417a8e8ef031d132afae` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:05.212932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab9dd8fb53c417a8e8ef031d132afae)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:05.214773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:05.478004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab9dd8fb53c417a8e8ef031d132afae'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.620569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab9dd8fb53c417a8e8ef031d132afae'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.629297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab9dd8fb53c417a8e8ef031d132afae'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.632970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab9dd8fb53c417a8e8ef031d132afae'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.636067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:05.638043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:05.655316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-43dad833-1eb3-4a02-8ab6-4feb58b7c8a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab9dd8fb53c417a8e8ef031d132afae'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.664509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1298/2541] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:05 2022] DELETE /identity/v3/users/1ab9dd8fb53c417a8e8ef031d132afae => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:05.665096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989bc8e-dad6-419e-9c2f-0324b8156790 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:05.726082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:05.727383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:05.727700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:05.728002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] PATH_INFO: `/v3/projects/e78a5a5d54134d3c937c3c0401301290` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:05.729257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78a5a5d54134d3c937c3c0401301290)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:05.731391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:05.878891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78a5a5d54134d3c937c3c0401301290', 'request_id': 'req-97680089-a9b3-48ac-8098-09a5ac873d0e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.974326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78a5a5d54134d3c937c3c0401301290', 'request_id': 'req-97680089-a9b3-48ac-8098-09a5ac873d0e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.985139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78a5a5d54134d3c937c3c0401301290', 'request_id': 'req-97680089-a9b3-48ac-8098-09a5ac873d0e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.995769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] The token cache is being invalidate because project e78a5a5d54134d3c937c3c0401301290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:05.996082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.996738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-97680089-a9b3-48ac-8098-09a5ac873d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:05.998569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1244/2542] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:05 2022] DELETE /identity/v3/projects/e78a5a5d54134d3c937c3c0401301290 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:06.004017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.008054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.008054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.025213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:06.026330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:06.086700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1299/2543] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:06.094609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feda509-da23-4fc6-98c8-201115f0334c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.095071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feda509-da23-4fc6-98c8-201115f0334c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.095389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feda509-da23-4fc6-98c8-201115f0334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.105892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.108067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:06.121171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2feda509-da23-4fc6-98c8-201115f0334c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:06.122574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2feda509-da23-4fc6-98c8-201115f0334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:06.126411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f406d-af78-42c4-ae80-ec248b98b8bf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.134820 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.145176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.145477 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.145759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] PATH_INFO: `/v3/projects/9e1e4329a8b2409ab4e4bfa0ce4e0323` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.148011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1e4329a8b2409ab4e4bfa0ce4e0323)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.150285 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.154716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1245/2544] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:06.159080 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feda509-da23-4fc6-98c8-201115f0334c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.169943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feda509-da23-4fc6-98c8-201115f0334c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.176904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.178381 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.180595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.180876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.182016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:06.182345 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.184061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.197807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1246/2545] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:22:06.202175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.212424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336071c-b51c-46f8-bdcd-1ac63e600524 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.217175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.217826 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.218111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.218387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.219102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.221033 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.268757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1e4329a8b2409ab4e4bfa0ce4e0323', 'request_id': 'req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:06.274348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1e4329a8b2409ab4e4bfa0ce4e0323', 'request_id': 'req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:06.280860 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1e4329a8b2409ab4e4bfa0ce4e0323', 'request_id': 'req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:06.335688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1247/2546] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:06.339695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.359239 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] The token cache is being invalidate because project 9e1e4329a8b2409ab4e4bfa0ce4e0323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:06.369081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:06.369081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40ce2ff2-eed7-405b-b094-3ce8fdf735bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:06.369081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1300/2547] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:06 2022] DELETE /identity/v3/projects/9e1e4329a8b2409ab4e4bfa0ce4e0323 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:06.373260 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8056aa-8a66-4c18-9660-a2a54583fd1b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.373700 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8056aa-8a66-4c18-9660-a2a54583fd1b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.373988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8056aa-8a66-4c18-9660-a2a54583fd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.394255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7a8056aa-8a66-4c18-9660-a2a54583fd1b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:06.395443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7a8056aa-8a66-4c18-9660-a2a54583fd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:06.407140 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30182e-3558-4086-8eea-b8a15c09a72f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.411936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.412605 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.412888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.413168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.415029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.417003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.457589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1301/2548] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:06.466312 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37df2b-8670-45f1-8623-f106a0e16e30 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.466602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37df2b-8670-45f1-8623-f106a0e16e30 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.466908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37df2b-8670-45f1-8623-f106a0e16e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.488674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1c37df2b-8670-45f1-8623-f106a0e16e30 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:06.489891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1c37df2b-8670-45f1-8623-f106a0e16e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:06.503342 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1302/2549] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:06.504691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37df2b-8670-45f1-8623-f106a0e16e30 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.514540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37df2b-8670-45f1-8623-f106a0e16e30 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.519319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037f213-d744-4450-a08c-536581076117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.519988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f213-d744-4450-a08c-536581076117 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.520269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f213-d744-4450-a08c-536581076117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.520546 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f213-d744-4450-a08c-536581076117 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.521272 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037f213-d744-4450-a08c-536581076117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:06.521845 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037f213-d744-4450-a08c-536581076117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.523718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037f213-d744-4450-a08c-536581076117 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.532646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1248/2550] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:06.536145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1303/2551] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:22:06.539867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.539867 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:06.541534 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037f213-d744-4450-a08c-536581076117 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.551467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037f213-d744-4450-a08c-536581076117 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.551691 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb87c-744d-401f-9285-5057d6e8d3b7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.556737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.560104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.560229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.560229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.560398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.560496 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.560595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.560595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.560804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.560893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.560893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.561254 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.574945 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1249/2552] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:06.576248 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.586665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194fb5d-7474-4109-bb7a-b470a9558cb9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.592499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.593267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.593553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.593853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.594576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.596237 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.612113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1250/2553] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:06.612113 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.622161 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c4b6-3c55-4382-9d73-15c8ca5d8146 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.626836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b0e9d-3bf7-4272-b655-28db43c3f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.627762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b0e9d-3bf7-4272-b655-28db43c3f8c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.628047 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b0e9d-3bf7-4272-b655-28db43c3f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.628318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b0e9d-3bf7-4272-b655-28db43c3f8c3 admin admin] PATH_INFO: `/v3/projects/a3f9321b9e0e4d7199d0194607c3a0ea/users/45ee4a94724f4905bd08afd4a3f0768f/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.643934 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b0e9d-3bf7-4272-b655-28db43c3f8c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f9321b9e0e4d7199d0194607c3a0ea, user_id=45ee4a94724f4905bd08afd4a3f0768f, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.647162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b0e9d-3bf7-4272-b655-28db43c3f8c3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.669044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1304/2554] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:06.673261 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.733955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a11fac-aac7-4edd-99a2-1a26e50ba1b0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.738680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.739350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.739655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.739945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.740633 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.742867 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.758738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1251/2555] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:06 2022] PUT /identity/v3/projects/a3f9321b9e0e4d7199d0194607c3a0ea/users/45ee4a94724f4905bd08afd4a3f0768f/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:06.770387 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6890-b84b-494e-969e-0602e00cba73 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.770609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6890-b84b-494e-969e-0602e00cba73 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.770882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6890-b84b-494e-969e-0602e00cba73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.799453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c94f6890-b84b-494e-969e-0602e00cba73 None None] MFA Rules not processed for user `45ee4a94724f4905bd08afd4a3f0768f`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:06.800695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c94f6890-b84b-494e-969e-0602e00cba73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:06.867512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1252/2556] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:06.871908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6890-b84b-494e-969e-0602e00cba73 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.903791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1305/2557] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:06.904975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:06.916690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f99374-87a1-48b7-9329-c541c24daa07 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.935667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6890-b84b-494e-969e-0602e00cba73 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:06.940813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.941546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.941880 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.942155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.942837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.946001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.978086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:06.978786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:06.979073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:06.979357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:06.980165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:06.981832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:06.995446 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1306/2558] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:06.996786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.006935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adcb63-a342-4218-988a-0b9eb8339773 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.012817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.013479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.013770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.014104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.014849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.016564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.027520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1307/2559] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:07.031673 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.042584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d89601-0f39-4b3f-b94c-93697fd7c74b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.048811 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642c078-a175-4a8b-964e-44dcf72d1ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.051044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642c078-a175-4a8b-964e-44dcf72d1ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.051252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642c078-a175-4a8b-964e-44dcf72d1ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.051539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642c078-a175-4a8b-964e-44dcf72d1ea3 admin admin] PATH_INFO: `/v3/projects/f8110316a04949afbb214f4f46f34c77/users/ba4d468428d24fc4be20c8583692794a/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.058530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1253/2560] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:06 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:07.060261 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.072809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642c078-a175-4a8b-964e-44dcf72d1ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8110316a04949afbb214f4f46f34c77, user_id=ba4d468428d24fc4be20c8583692794a, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.075689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642c078-a175-4a8b-964e-44dcf72d1ea3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.125522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3fe77-b1f7-4cbe-b428-414e4b44eb6e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.130301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.130971 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.131255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.131532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.132824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.134755 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.155726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1308/2561] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:07 2022] PUT /identity/v3/projects/f8110316a04949afbb214f4f46f34c77/users/ba4d468428d24fc4be20c8583692794a/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:07.167849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08da4-e451-4089-b3a5-e0778fc7709f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.168236 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08da4-e451-4089-b3a5-e0778fc7709f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.168586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08da4-e451-4089-b3a5-e0778fc7709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.199541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-88d08da4-e451-4089-b3a5-e0778fc7709f None None] MFA Rules not processed for user `ba4d468428d24fc4be20c8583692794a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:07.200880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-88d08da4-e451-4089-b3a5-e0778fc7709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:07.208553 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1254/2562] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:07 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:07.210929 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.228155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e6f6-95ce-4132-98aa-85265a2a45b9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.279144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1309/2563] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:07.280151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08da4-e451-4089-b3a5-e0778fc7709f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.291523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.292239 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.292520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.292792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.293639 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.295393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.306479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1255/2564] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:07.310531 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.321306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fbca7-0062-4ddf-95de-f67ce8c01966 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.332524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.332524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.332524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.332524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] PATH_INFO: `/v3/projects/df23d918e9e84e40b2c38a114bcd0adf/users/0ade79aa1ba6463aa38caafab0076833/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.354315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df23d918e9e84e40b2c38a114bcd0adf, user_id=0ade79aa1ba6463aa38caafab0076833, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.357180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.368130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08da4-e451-4089-b3a5-e0778fc7709f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.372948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.373669 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.378146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.378146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.378146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.378146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.515368 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1256/2565] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:07 2022] PUT /identity/v3/projects/df23d918e9e84e40b2c38a114bcd0adf/users/0ade79aa1ba6463aa38caafab0076833/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:07.516307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.531933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1310/2566] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:07 2022] POST /identity/v3/projects => generated 369 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:07.531933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.531933 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:07.594235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a602f-aa0a-4ef1-a64d-04a3b8c38a25 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.615082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49f7af-9344-480e-ae5b-7dc7e0baf70a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.621833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.622717 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.623000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.623278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] PATH_INFO: `/v3/users/5d59e244dceb4d25a4bbe888c9a42002` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.624520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d59e244dceb4d25a4bbe888c9a42002)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.626526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.663251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.664328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.664626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.664907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.665647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.667295 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.677943 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1257/2567] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:07.680581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.692557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34249b30-456c-4e57-b850-526fa135f2a6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.698010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.698686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.699096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.699383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.700127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.702073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.806927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1258/2568] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:07 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:07.806927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.816236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1933d1b-02e1-472f-b3d8-7cadeb7964ec admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.822146 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.823059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.823328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.823611 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] PATH_INFO: `/v3/projects/df23d918e9e84e40b2c38a114bcd0adf/users/0ade79aa1ba6463aa38caafab0076833/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.826766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df23d918e9e84e40b2c38a114bcd0adf, user_id=0ade79aa1ba6463aa38caafab0076833, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.828791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.850704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1259/2569] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:07 2022] PUT /identity/v3/projects/df23d918e9e84e40b2c38a114bcd0adf/users/0ade79aa1ba6463aa38caafab0076833/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:07.850704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:07.871662 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d59e244dceb4d25a4bbe888c9a42002'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:07.917374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2d49c-76a3-4556-92f6-c3ab7318a779 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:07.923217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7beb1a-9114-4a33-8fe8-20136a7a873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:07.923217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7beb1a-9114-4a33-8fe8-20136a7a873e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.923217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7beb1a-9114-4a33-8fe8-20136a7a873e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.923729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7beb1a-9114-4a33-8fe8-20136a7a873e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.924064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7beb1a-9114-4a33-8fe8-20136a7a873e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:07.925651 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7beb1a-9114-4a33-8fe8-20136a7a873e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:07.936892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1260/2570] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:07.938180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262e08d-ef9a-484f-b25a-ee6fb9e33093 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:07.938468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262e08d-ef9a-484f-b25a-ee6fb9e33093 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:07.938746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262e08d-ef9a-484f-b25a-ee6fb9e33093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:07.970672 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5262e08d-ef9a-484f-b25a-ee6fb9e33093 None None] MFA Rules not processed for user `0ade79aa1ba6463aa38caafab0076833`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:07.971517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d59e244dceb4d25a4bbe888c9a42002'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:07.971920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5262e08d-ef9a-484f-b25a-ee6fb9e33093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:07.980902 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d59e244dceb4d25a4bbe888c9a42002'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:07.981831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d59e244dceb4d25a4bbe888c9a42002'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:07.995989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d59e244dceb4d25a4bbe888c9a42002'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:08.003213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1311/2571] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:07 2022] DELETE /identity/v3/users/5d59e244dceb4d25a4bbe888c9a42002 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:08.006921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:08.009595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:08.055936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1261/2572] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:08.055936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.055936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.055936 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.080296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:08.081548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:08.084955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49340bf9-f64c-44a2-bb3d-85a82dc83815 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:08.093683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1262/2573] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:08.093884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:08.094016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:08.094102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.094102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.094102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] PATH_INFO: `/v3/projects/9522eb9c52184a39a8deb29ec4141ede/users/fde7557d9b5d4abc9f33a7e39287a4f4/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.094792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:08.116292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9522eb9c52184a39a8deb29ec4141ede, user_id=fde7557d9b5d4abc9f33a7e39287a4f4, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:08.121656 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:08.156005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005265d-1a26-41c1-8bab-626fc65ce9c2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:08.159825 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:08.160516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.160799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.161076 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.206548 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1312/2574] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:08 2022] PUT /identity/v3/projects/9522eb9c52184a39a8deb29ec4141ede/users/fde7557d9b5d4abc9f33a7e39287a4f4/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:08.207762 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:08.210103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:08.227073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:08.228636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:08.238875 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1263/2575] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:08.239986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:08.252372 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ae3a6-1a90-46df-a698-1a57c9fe9ba7 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:08.275427 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01774438-6b16-4641-9079-d57a68789b49 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:08.279963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:08.280674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.281007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.281776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.314249 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:08.314988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.315332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.315831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.316543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:08.318389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:08.334704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1264/2576] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:08.334704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:08.352183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a571720-830a-4808-96b1-c04ea3a1056d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:08.357744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24afdc9-1e9b-4aac-a8db-4d92b1a2f1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:08.358834 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24afdc9-1e9b-4aac-a8db-4d92b1a2f1be admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.359507 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24afdc9-1e9b-4aac-a8db-4d92b1a2f1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.360170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24afdc9-1e9b-4aac-a8db-4d92b1a2f1be admin admin] PATH_INFO: `/v3/projects/9522eb9c52184a39a8deb29ec4141ede/users/fde7557d9b5d4abc9f33a7e39287a4f4/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.363154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24afdc9-1e9b-4aac-a8db-4d92b1a2f1be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9522eb9c52184a39a8deb29ec4141ede, user_id=fde7557d9b5d4abc9f33a7e39287a4f4, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:08.367056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24afdc9-1e9b-4aac-a8db-4d92b1a2f1be admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:08.383207 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1265/2577] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:08 2022] PUT /identity/v3/projects/9522eb9c52184a39a8deb29ec4141ede/users/fde7557d9b5d4abc9f33a7e39287a4f4/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:08.389138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:08.391712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:08.397225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.397623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.398020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.410275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1313/2578] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:08.434376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] MFA Rules not processed for user `fde7557d9b5d4abc9f33a7e39287a4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:08.435909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:08.534448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1266/2579] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:08.550434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:08.551947 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:08.629866 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048625b-16cd-4b59-8cb8-8997a892026b service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:08.732555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:08.733492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:08.733783 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:08.734067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:08.756435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:08.758228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:08.769422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1314/2580] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:08 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:08.949457 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:08.952932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:09.061250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:09.068247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013d182-f362-4cad-a908-e8fca10cf9c9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:09.073044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b14391-ce06-4503-b8ea-7b3b4cf9192a service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:09.075014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:09.076739 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:09.077519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:09.077960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] PATH_INFO: `/v3/projects/46876a2b970f44cf871a1ff42ee6a06c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:09.079727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] RBAC: Authorizing `identity:delete_project(project_id=46876a2b970f44cf871a1ff42ee6a06c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:09.082025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:09.090122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:09.090122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:09.090122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:09.090122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:09.090582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:09.168851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:09.170642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:09.183739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1315/2581] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:09.187604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46876a2b970f44cf871a1ff42ee6a06c', 'request_id': 'req-79617f68-cef2-43d9-9506-ffb7306b1bec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.274598 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46876a2b970f44cf871a1ff42ee6a06c', 'request_id': 'req-79617f68-cef2-43d9-9506-ffb7306b1bec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.281000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46876a2b970f44cf871a1ff42ee6a06c', 'request_id': 'req-79617f68-cef2-43d9-9506-ffb7306b1bec'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.293791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] The token cache is being invalidate because project 46876a2b970f44cf871a1ff42ee6a06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:09.294084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.294732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.296359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1267/2582] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:08 2022] DELETE /identity/v3/projects/46876a2b970f44cf871a1ff42ee6a06c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:09.305404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:09.310104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:09.311762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:09.313487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90352b-8748-4571-b28b-3734bb7f3c01 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:09.373248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:09.374140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:09.374425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:09.374699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] PATH_INFO: `/v3/users/3cdba89fe20c4e4a914b81e6aaa72691` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:09.375926 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cdba89fe20c4e4a914b81e6aaa72691)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:09.378034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:09.379179 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79617f68-cef2-43d9-9506-ffb7306b1bec admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:09.385102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:09.386470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:09.386470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:09.386633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] PATH_INFO: `/v3/users/45ee4a94724f4905bd08afd4a3f0768f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:09.387784 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ee4a94724f4905bd08afd4a3f0768f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:09.389736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:09.624550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ee4a94724f4905bd08afd4a3f0768f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.642893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cdba89fe20c4e4a914b81e6aaa72691'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.694165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ee4a94724f4905bd08afd4a3f0768f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.695044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ee4a94724f4905bd08afd4a3f0768f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.708808 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ee4a94724f4905bd08afd4a3f0768f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.716872 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf6517c4-70cd-49d7-8763-f1eaec5d7790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ee4a94724f4905bd08afd4a3f0768f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.723703 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1268/2583] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:09 2022] DELETE /identity/v3/users/45ee4a94724f4905bd08afd4a3f0768f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:09.728430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:09.732180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:09.732180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:09.748356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:09.749736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:09.779980 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cdba89fe20c4e4a914b81e6aaa72691'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.787806 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cdba89fe20c4e4a914b81e6aaa72691'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.789555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cdba89fe20c4e4a914b81e6aaa72691'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.803726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cdba89fe20c4e4a914b81e6aaa72691'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:09.810694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1316/2584] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:09 2022] DELETE /identity/v3/users/3cdba89fe20c4e4a914b81e6aaa72691 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:09.813759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1269/2585] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:09.829718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:09.831199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:09.898999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51180fa-86a7-4998-a9d8-7a409f006002 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:09.903243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:09.903919 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:09.904200 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:09.904469 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:09.963773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:09.965252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:09.974632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1317/2586] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:10.219761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.221278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:10.233400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174041ff-2731-46cd-a327-25eeb57eaa67 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.238602 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.239198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.239474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.239782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] PATH_INFO: `/v3/projects/a8a8080a0e1d43d1a87f6aec46723372` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.240976 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a8080a0e1d43d1a87f6aec46723372)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.243480 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.354464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a8080a0e1d43d1a87f6aec46723372', 'request_id': 'req-1c9b2509-8509-4a12-914c-4a15a761e62b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.370830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.372283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:10.388438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11e557-c7f2-47f3-99b1-25ced6d2f095 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.408767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a8080a0e1d43d1a87f6aec46723372', 'request_id': 'req-1c9b2509-8509-4a12-914c-4a15a761e62b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.414982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a8080a0e1d43d1a87f6aec46723372', 'request_id': 'req-1c9b2509-8509-4a12-914c-4a15a761e62b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.426887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] The token cache is being invalidate because project a8a8080a0e1d43d1a87f6aec46723372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:10.427156 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.427827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1c9b2509-8509-4a12-914c-4a15a761e62b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.429957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1270/2587] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:10 2022] DELETE /identity/v3/projects/a8a8080a0e1d43d1a87f6aec46723372 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:10.434846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea607d-fee8-4531-9b8d-1cd31fb569a5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.435138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea607d-fee8-4531-9b8d-1cd31fb569a5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.435421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea607d-fee8-4531-9b8d-1cd31fb569a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.448471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.449375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.449658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.449931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] PATH_INFO: `/v3/projects/a3f9321b9e0e4d7199d0194607c3a0ea` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.454585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f9321b9e0e4d7199d0194607c3a0ea)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.454585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.454791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e3ea607d-fee8-4531-9b8d-1cd31fb569a5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:10.454988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e3ea607d-fee8-4531-9b8d-1cd31fb569a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:10.463716 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1271/2588] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:10.471253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e456d-a798-433f-9d69-bde14a2a555b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.471545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e456d-a798-433f-9d69-bde14a2a555b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.471859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e456d-a798-433f-9d69-bde14a2a555b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.488913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6f4e456d-a798-433f-9d69-bde14a2a555b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:10.490110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6f4e456d-a798-433f-9d69-bde14a2a555b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:10.498695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1272/2589] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:10.502621 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e456d-a798-433f-9d69-bde14a2a555b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.511400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e456d-a798-433f-9d69-bde14a2a555b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.515969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.516591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.516876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.517152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.517855 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:10.518162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.519772 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.531939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1273/2590] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:22:10.535933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.545906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a49c2-8383-4d79-9e3f-2d462e664bd7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.551115 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f9321b9e0e4d7199d0194607c3a0ea', 'request_id': 'req-290f3260-724e-43e6-b062-ef4e1e54ed85'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.569194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f9321b9e0e4d7199d0194607c3a0ea', 'request_id': 'req-290f3260-724e-43e6-b062-ef4e1e54ed85'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.575815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f9321b9e0e4d7199d0194607c3a0ea', 'request_id': 'req-290f3260-724e-43e6-b062-ef4e1e54ed85'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.600300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.601646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.602111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.602388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.603064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.604970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.654256 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] The token cache is being invalidate because project a3f9321b9e0e4d7199d0194607c3a0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:10.654541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.655152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.656734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1318/2591] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:10 2022] DELETE /identity/v3/projects/a3f9321b9e0e4d7199d0194607c3a0ea => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:10.660594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.661904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:10.674755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f3260-724e-43e6-b062-ef4e1e54ed85 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.681523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.681523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.681523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.681523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] PATH_INFO: `/v3/users/0ade79aa1ba6463aa38caafab0076833` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.682000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ade79aa1ba6463aa38caafab0076833)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.683990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.705766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1274/2592] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:10 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:10.715927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.715927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:10.774856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5481f-73e9-44ae-83e6-bcff74eec85b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.779724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.780394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.780681 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.780954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.781641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.783547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.852788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ade79aa1ba6463aa38caafab0076833'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.867686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1275/2593] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:10 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:10.874807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.885424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d991-d7c6-43d8-965f-5e1d473ff513 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.890267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.894310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.894310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.894310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.894310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.894310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.905066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1276/2594] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:10.908799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.911364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ade79aa1ba6463aa38caafab0076833'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.918767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58379b-0007-4d3a-9906-87f144606751 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:10.920797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ade79aa1ba6463aa38caafab0076833'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.920797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ade79aa1ba6463aa38caafab0076833'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.923831 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:10.924483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:10.924769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:10.925035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:10.925751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:10.927332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:10.935309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e37c71bd-a5b0-40af-9373-e5d3877a8a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ade79aa1ba6463aa38caafab0076833'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:10.942025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1277/2596] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:10.946129 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1319/2596] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:10 2022] DELETE /identity/v3/users/0ade79aa1ba6463aa38caafab0076833 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:10.946259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:10.947989 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:10.963178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2f6de-c076-4735-9cfe-26bddbf6a7d4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.020383 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.021285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.022037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.022151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] PATH_INFO: `/v3/projects/5625fcdcdb074ecf9c4b74060feabb68/users/dd8871a4e23a44db88f845ee1b823688/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.038695 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5625fcdcdb074ecf9c4b74060feabb68, user_id=dd8871a4e23a44db88f845ee1b823688, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.041512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.152219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1278/2597] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:10 2022] PUT /identity/v3/projects/5625fcdcdb074ecf9c4b74060feabb68/users/dd8871a4e23a44db88f845ee1b823688/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:11.164064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4690c-bb65-4030-9039-562734576653 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.164356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4690c-bb65-4030-9039-562734576653 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.164635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4690c-bb65-4030-9039-562734576653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.191645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d5a4690c-bb65-4030-9039-562734576653 None None] MFA Rules not processed for user `dd8871a4e23a44db88f845ee1b823688`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:11.192959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d5a4690c-bb65-4030-9039-562734576653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:11.261542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1320/2598] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:11.266071 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.327679 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2342ac-1e79-4209-996e-188153445651 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.334614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.335471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.335901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.336365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.337122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.339104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.379274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4690c-bb65-4030-9039-562734576653 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.380771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d5a4690c-bb65-4030-9039-562734576653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:11.400238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4690c-bb65-4030-9039-562734576653 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.406282 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.407445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.408338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.408338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] PATH_INFO: `/v3/projects/df23d918e9e84e40b2c38a114bcd0adf` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.409904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df23d918e9e84e40b2c38a114bcd0adf)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.412585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.459671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1279/2599] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:11.463819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.523419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae3e06-ba91-465b-beb0-031f7a3442e0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.528181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.528848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.529129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.529403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.530109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.532059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.537903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df23d918e9e84e40b2c38a114bcd0adf', 'request_id': 'req-699d9908-8771-46c1-9233-2489ee5dff7d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:11.544441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df23d918e9e84e40b2c38a114bcd0adf', 'request_id': 'req-699d9908-8771-46c1-9233-2489ee5dff7d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:11.550927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df23d918e9e84e40b2c38a114bcd0adf', 'request_id': 'req-699d9908-8771-46c1-9233-2489ee5dff7d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:11.620876 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1280/2600] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:11.624600 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.634034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95063f0-3667-4402-85e8-d923b7028166 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.662167 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] The token cache is being invalidate because project df23d918e9e84e40b2c38a114bcd0adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:11.662457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:11.663099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-699d9908-8771-46c1-9233-2489ee5dff7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:11.665215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1321/2601] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:11 2022] DELETE /identity/v3/projects/df23d918e9e84e40b2c38a114bcd0adf => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:11.673725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4e457-53a5-434a-a242-a344e02835c1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.673877 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4e457-53a5-434a-a242-a344e02835c1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.674064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4e457-53a5-434a-a242-a344e02835c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.691932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.692617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.692773 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9de4e457-53a5-434a-a242-a344e02835c1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:11.692885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.693145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.693870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.693966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9de4e457-53a5-434a-a242-a344e02835c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:11.698625 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.702526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1322/2602] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:11.709417 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1281/2603] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:11.711104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.711233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93544d8b-4002-47e0-ada0-1194d62db21d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.711430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93544d8b-4002-47e0-ada0-1194d62db21d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.711727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93544d8b-4002-47e0-ada0-1194d62db21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.712948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:11.725155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880f457-dcfa-4950-b20b-880e78473c61 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.729900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-93544d8b-4002-47e0-ada0-1194d62db21d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:11.730919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.731102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-93544d8b-4002-47e0-ada0-1194d62db21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:11.731824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.732106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.732375 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] PATH_INFO: `/v3/projects/6a1c09979bbf417c9bd76107c098059e/users/80c56b16b3b243e8a394fea6d90ef792/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.739246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1323/2604] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:11.744073 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93544d8b-4002-47e0-ada0-1194d62db21d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.748413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1c09979bbf417c9bd76107c098059e, user_id=80c56b16b3b243e8a394fea6d90ef792, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.751091 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.752982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93544d8b-4002-47e0-ada0-1194d62db21d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.757561 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.758185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.758466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.758745 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.765335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:11.765335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.765335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.772728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1324/2605] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:22:11.776823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.785966 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ab111-eead-4d07-82bd-53530d0854c5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.791056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.791698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.791978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.792249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.792913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.794755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.811862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1282/2606] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:11 2022] PUT /identity/v3/projects/6a1c09979bbf417c9bd76107c098059e/users/80c56b16b3b243e8a394fea6d90ef792/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:11.815763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.876267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5999921-f46f-4989-8177-a2ff0f0425c8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.878321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1325/2607] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:11.882310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.933653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.934352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.934647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.935034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.935802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.937399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.946037 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb981e-aa24-42a1-85c5-c2016e9089d6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.947862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1283/2608] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:11.954151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.954151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.954151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.954509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:11.954614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.954614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.954889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.961920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ecf0b-5465-4a0b-af05-88bbf889025e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:11.966597 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe5324-2648-4d91-bc5a-650aff0b8c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:11.967481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5324-2648-4d91-bc5a-650aff0b8c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.967792 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5324-2648-4d91-bc5a-650aff0b8c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.968078 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5324-2648-4d91-bc5a-650aff0b8c65 admin admin] PATH_INFO: `/v3/projects/6a1c09979bbf417c9bd76107c098059e/users/80c56b16b3b243e8a394fea6d90ef792/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:11.969981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe5324-2648-4d91-bc5a-650aff0b8c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1c09979bbf417c9bd76107c098059e, user_id=80c56b16b3b243e8a394fea6d90ef792, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:11.974603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe5324-2648-4d91-bc5a-650aff0b8c65 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:11.982704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1284/2609] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:11 2022] PUT /identity/v3/projects/6a1c09979bbf417c9bd76107c098059e/users/80c56b16b3b243e8a394fea6d90ef792/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:11.993510 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:11.997520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:11.997520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.020944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] MFA Rules not processed for user `80c56b16b3b243e8a394fea6d90ef792`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:12.022422 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:12.054698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1326/2610] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:12.055958 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.065861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a58ae-0ac1-4a5b-98ec-a64d17507739 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.097899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1285/2611] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:12.109388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.110829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:12.126327 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.127192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.127577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.127967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.128731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.130308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.141317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1327/2612] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:12.144880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.155162 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d84823-91b3-4b72-82b5-a406f5c4fec7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.165710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.165710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.165710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.165710 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.166083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.166083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.174350 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1328/2613] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:12.174877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151f11f-a0ba-4f71-9e6d-4de6297467a9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.178660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fba9cb-be63-4a8b-82b1-519deb91a0d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.179887 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba9cb-be63-4a8b-82b1-519deb91a0d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.180171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba9cb-be63-4a8b-82b1-519deb91a0d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.180341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.180459 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba9cb-be63-4a8b-82b1-519deb91a0d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.189595 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fba9cb-be63-4a8b-82b1-519deb91a0d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.189737 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd7be-5053-4dfe-87a5-336409d641f6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.191035 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fba9cb-be63-4a8b-82b1-519deb91a0d3 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.194343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.195324 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.195602 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.195908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] PATH_INFO: `/v3/projects/ca980e6b85ae4184b8056e1af9b8b6d2/users/fde99aff2ca940fbb8c6c90443b51e36/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.200693 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1286/2614] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:12.210722 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca980e6b85ae4184b8056e1af9b8b6d2, user_id=fde99aff2ca940fbb8c6c90443b51e36, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.213394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.274217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1329/2615] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:12 2022] PUT /identity/v3/projects/ca980e6b85ae4184b8056e1af9b8b6d2/users/fde99aff2ca940fbb8c6c90443b51e36/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:12.286005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a7c7c-9430-408c-bed3-91e2c7fca356 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.286272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a7c7c-9430-408c-bed3-91e2c7fca356 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.286578 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a7c7c-9430-408c-bed3-91e2c7fca356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.320571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-058a7c7c-9430-408c-bed3-91e2c7fca356 None None] MFA Rules not processed for user `fde99aff2ca940fbb8c6c90443b51e36`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:12.321854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-058a7c7c-9430-408c-bed3-91e2c7fca356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:12.389172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1287/2616] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:12.393541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.461879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b386f2-a6d3-469a-acba-d762cf08268b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.466963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.467654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.467927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.468198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.468910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.470785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.557603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1330/2617] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:12 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:12.560411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a7c7c-9430-408c-bed3-91e2c7fca356 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.571622 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a7c7c-9430-408c-bed3-91e2c7fca356 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.628775 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.629435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.629715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.629991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.630674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.632762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.714319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.719006 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1288/2618] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:12 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:12.722744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.724535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3d722-1a85-4d91-9209-735acbac227b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.729855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.730740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.730995 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.731270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] PATH_INFO: `/v3/users/dd8871a4e23a44db88f845ee1b823688` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.732487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8871a4e23a44db88f845ee1b823688)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.733590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb85f6-20dd-442c-bd62-eeae3bfeb538 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.734515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.738310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.739014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.739299 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.739571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.740320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.741918 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.754659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1289/2619] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:12.755747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.767428 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91fbb1-f337-4730-84bf-f7bcb0db654b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.772670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.774001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.774284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.774560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] PATH_INFO: `/v3/projects/c94ae79784be4fddbd4b85933d4bcf5b/users/7777b69974724fcfa7beece98840693f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.789204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94ae79784be4fddbd4b85933d4bcf5b, user_id=7777b69974724fcfa7beece98840693f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.791964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.872512 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1290/2620] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:12 2022] PUT /identity/v3/projects/c94ae79784be4fddbd4b85933d4bcf5b/users/7777b69974724fcfa7beece98840693f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:12.876201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.916461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8871a4e23a44db88f845ee1b823688'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:12.937224 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc84f4-248a-4f5e-b574-6abf76a0b064 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.941823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.942475 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.942752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.943024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.943760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.946003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.955894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1291/2621] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:12.959717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:12.969769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03d9de-fe02-42f2-8236-c1befe13c1fa admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:12.971554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8871a4e23a44db88f845ee1b823688'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:12.974350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808267bc-ac43-428a-a230-801ed8937198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:12.975240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808267bc-ac43-428a-a230-801ed8937198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:12.980393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808267bc-ac43-428a-a230-801ed8937198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:12.980393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808267bc-ac43-428a-a230-801ed8937198 admin admin] PATH_INFO: `/v3/projects/c94ae79784be4fddbd4b85933d4bcf5b/users/7777b69974724fcfa7beece98840693f/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:12.980393 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808267bc-ac43-428a-a230-801ed8937198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94ae79784be4fddbd4b85933d4bcf5b, user_id=7777b69974724fcfa7beece98840693f, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:12.980680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8871a4e23a44db88f845ee1b823688'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:12.980680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8871a4e23a44db88f845ee1b823688'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:12.980866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808267bc-ac43-428a-a230-801ed8937198 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:12.990636 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1292/2622] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:12 2022] PUT /identity/v3/projects/c94ae79784be4fddbd4b85933d4bcf5b/users/7777b69974724fcfa7beece98840693f/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:12.992665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9ad2d5d8-2d8d-42c7-9ee6-978f16c0ee24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8871a4e23a44db88f845ee1b823688'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:12.998741 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1331/2623] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:12 2022] DELETE /identity/v3/users/dd8871a4e23a44db88f845ee1b823688 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:13.000657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.000951 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.001232 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.003056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.003348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.003699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.021090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:13.022559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.029823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] MFA Rules not processed for user `7777b69974724fcfa7beece98840693f`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:13.031049 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.090415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1332/2624] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:13.100289 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:13.101852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.107001 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1293/2625] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:13.122845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:13.124431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.159532 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a795-0e32-4a83-824d-fd6fb8cbc92a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:13.163242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:13.164343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.164852 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.165360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.177620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:13.179094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:13.182007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e5a64-1db6-4b8f-afdc-b683b9851b07 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:13.186044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63307356-71c5-4ae7-a456-06f67c6b36e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:13.186624 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307356-71c5-4ae7-a456-06f67c6b36e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.186903 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307356-71c5-4ae7-a456-06f67c6b36e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.187173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307356-71c5-4ae7-a456-06f67c6b36e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.190904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1333/2626] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:13.197791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63307356-71c5-4ae7-a456-06f67c6b36e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:13.199067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63307356-71c5-4ae7-a456-06f67c6b36e2 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:13.215104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1294/2627] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:13.286638 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:13.292386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.306974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1607b3f-ce10-44a2-80c3-4a7c40834a2e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:13.315880 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:13.317006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.317351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.317691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] PATH_INFO: `/v3/users/fde99aff2ca940fbb8c6c90443b51e36` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.319142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde99aff2ca940fbb8c6c90443b51e36)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:13.322766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:13.530979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde99aff2ca940fbb8c6c90443b51e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.582625 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde99aff2ca940fbb8c6c90443b51e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.589948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde99aff2ca940fbb8c6c90443b51e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.590591 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde99aff2ca940fbb8c6c90443b51e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.601916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde99aff2ca940fbb8c6c90443b51e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.607654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1334/2628] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:13 2022] DELETE /identity/v3/users/fde99aff2ca940fbb8c6c90443b51e36 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:13.612090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c699b-5240-49e3-9661-c003666c327b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.612557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c699b-5240-49e3-9661-c003666c327b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.612652 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c699b-5240-49e3-9661-c003666c327b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.629804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-478c699b-5240-49e3-9661-c003666c327b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:13.631008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-478c699b-5240-49e3-9661-c003666c327b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.636379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:13.637913 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.698063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1295/2629] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:13.707430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c699b-5240-49e3-9661-c003666c327b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:13.708006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b05d2-2ace-4d21-b97f-e5b890f7fae7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:13.708930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-478c699b-5240-49e3-9661-c003666c327b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:13.713639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:13.714542 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.715011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.715294 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] PATH_INFO: `/v3/projects/5625fcdcdb074ecf9c4b74060feabb68` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.716594 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5625fcdcdb074ecf9c4b74060feabb68)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:13.718555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:13.766090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c699b-5240-49e3-9661-c003666c327b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:13.769840 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:13.770488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.770771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.771043 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.780802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:13.782225 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:13.791549 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1296/2630] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:13.820241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5625fcdcdb074ecf9c4b74060feabb68', 'request_id': 'req-318088e6-943a-4864-9f16-3ea07f687cee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.825789 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5625fcdcdb074ecf9c4b74060feabb68', 'request_id': 'req-318088e6-943a-4864-9f16-3ea07f687cee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.831970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5625fcdcdb074ecf9c4b74060feabb68', 'request_id': 'req-318088e6-943a-4864-9f16-3ea07f687cee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.910608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] The token cache is being invalidate because project 5625fcdcdb074ecf9c4b74060feabb68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:13.910897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.911526 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:13.913119 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1335/2631] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:13 2022] DELETE /identity/v3/projects/5625fcdcdb074ecf9c4b74060feabb68 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:13.917215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:13.926820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e334a3-0801-43b2-b7b4-677148107b02 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:13.982623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:13.983589 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:13.983895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:13.984174 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] PATH_INFO: `/v3/users/80c56b16b3b243e8a394fea6d90ef792` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:13.985358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c56b16b3b243e8a394fea6d90ef792)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:13.987272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:14.170759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c56b16b3b243e8a394fea6d90ef792'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.187989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:14.189478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:14.202976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318088e6-943a-4864-9f16-3ea07f687cee admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:14.207985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:14.208850 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:14.209134 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:14.209416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] PATH_INFO: `/v3/projects/ca980e6b85ae4184b8056e1af9b8b6d2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:14.210668 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca980e6b85ae4184b8056e1af9b8b6d2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:14.212899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:14.234136 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c56b16b3b243e8a394fea6d90ef792'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.234777 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c56b16b3b243e8a394fea6d90ef792'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.247754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c56b16b3b243e8a394fea6d90ef792'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.254362 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c56b16b3b243e8a394fea6d90ef792'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.261398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1297/2632] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:13 2022] DELETE /identity/v3/users/80c56b16b3b243e8a394fea6d90ef792 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:14.321663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca980e6b85ae4184b8056e1af9b8b6d2', 'request_id': 'req-68eeea2d-57f7-48ad-a85d-971d4438e70b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.326989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca980e6b85ae4184b8056e1af9b8b6d2', 'request_id': 'req-68eeea2d-57f7-48ad-a85d-971d4438e70b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.333347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca980e6b85ae4184b8056e1af9b8b6d2', 'request_id': 'req-68eeea2d-57f7-48ad-a85d-971d4438e70b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.400253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] The token cache is being invalidate because project ca980e6b85ae4184b8056e1af9b8b6d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:14.400541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.401140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.402679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1336/2633] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:14 2022] DELETE /identity/v3/projects/ca980e6b85ae4184b8056e1af9b8b6d2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:14.406645 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:14.408081 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:14.493407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd7ff6-79c0-4a14-905b-1d04f7df1436 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:14.553215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:14.554230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:14.554886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:14.554886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] PATH_INFO: `/v3/users/7777b69974724fcfa7beece98840693f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:14.556013 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7777b69974724fcfa7beece98840693f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:14.557922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:14.702061 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:14.703486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:14.716568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeea2d-57f7-48ad-a85d-971d4438e70b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:14.721101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:14.722002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:14.722286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:14.722559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] PATH_INFO: `/v3/projects/6a1c09979bbf417c9bd76107c098059e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:14.723772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1c09979bbf417c9bd76107c098059e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:14.725686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:14.763982 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7777b69974724fcfa7beece98840693f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.864301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1c09979bbf417c9bd76107c098059e', 'request_id': 'req-8635d698-fd0a-46be-891f-49a819a416b1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.873230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1c09979bbf417c9bd76107c098059e', 'request_id': 'req-8635d698-fd0a-46be-891f-49a819a416b1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.874187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7777b69974724fcfa7beece98840693f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.875141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7777b69974724fcfa7beece98840693f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.880287 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1c09979bbf417c9bd76107c098059e', 'request_id': 'req-8635d698-fd0a-46be-891f-49a819a416b1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.889771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7777b69974724fcfa7beece98840693f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.896900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-aa9df24e-39fb-4dfb-9bbd-25835e1281e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7777b69974724fcfa7beece98840693f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.903376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1298/2634] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:14 2022] DELETE /identity/v3/users/7777b69974724fcfa7beece98840693f => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:14.970792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] The token cache is being invalidate because project 6a1c09979bbf417c9bd76107c098059e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:14.971108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.971784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8635d698-fd0a-46be-891f-49a819a416b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:14.973358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1337/2635] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:14 2022] DELETE /identity/v3/projects/6a1c09979bbf417c9bd76107c098059e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:14.979389 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76810bd-ede9-4f8d-96dd-778c9c690274 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:14.980048 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76810bd-ede9-4f8d-96dd-778c9c690274 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:14.980334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76810bd-ede9-4f8d-96dd-778c9c690274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:14.998931 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a76810bd-ede9-4f8d-96dd-778c9c690274 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:15.000178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a76810bd-ede9-4f8d-96dd-778c9c690274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:15.059172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1299/2636] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:15.066939 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a9e13-89bd-458f-bbc1-40b1cddb8b3b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.067227 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a9e13-89bd-458f-bbc1-40b1cddb8b3b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.067506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a9e13-89bd-458f-bbc1-40b1cddb8b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.086090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d01a9e13-89bd-458f-bbc1-40b1cddb8b3b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:15.087268 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d01a9e13-89bd-458f-bbc1-40b1cddb8b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:15.095170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1338/2637] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:15.099519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76810bd-ede9-4f8d-96dd-778c9c690274 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.115205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76810bd-ede9-4f8d-96dd-778c9c690274 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.117818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.118717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.118914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.119181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.119946 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:15.120246 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.121900 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.137511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1300/2638] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:22:15.141886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a9e13-89bd-458f-bbc1-40b1cddb8b3b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.153467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a9e13-89bd-458f-bbc1-40b1cddb8b3b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.158495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.159158 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.159442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.159750 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.160448 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.162351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.253080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1339/2639] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:15.257277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.267209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa14aae-4e5d-4c6d-b755-073f6488e57d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.314586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.316432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:15.328280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.343310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.343310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.343310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.343310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.343310 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.389614 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b69a-8b7f-4b95-9b83-b776fd9211f8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.394567 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.396643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.396643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.396643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] PATH_INFO: `/v3/projects/c94ae79784be4fddbd4b85933d4bcf5b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.398412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94ae79784be4fddbd4b85933d4bcf5b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.400458 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.432724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1301/2640] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:15.436908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.447863 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b61a15-2dd5-4227-a980-c0afade240f8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.452629 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.453398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.453674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.453949 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.454685 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.456344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.468117 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1302/2641] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:22:15.472110 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.482490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3f73b-59cc-426f-9aed-6aa64ba3896b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.487220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.488712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.488712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.488712 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.489176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.490791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.502253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1303/2642] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:15.506279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.516499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4959d10-7700-4ccb-8535-49e17c6541a7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.527341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94ae79784be4fddbd4b85933d4bcf5b', 'request_id': 'req-9e8788ea-4b1d-4913-85ab-1afe1a126281'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:15.532807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94ae79784be4fddbd4b85933d4bcf5b', 'request_id': 'req-9e8788ea-4b1d-4913-85ab-1afe1a126281'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:15.539117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94ae79784be4fddbd4b85933d4bcf5b', 'request_id': 'req-9e8788ea-4b1d-4913-85ab-1afe1a126281'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:15.577750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59209c95-0da3-458c-bdf5-9dc61588d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.578558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59209c95-0da3-458c-bdf5-9dc61588d47d admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.578841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59209c95-0da3-458c-bdf5-9dc61588d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.579114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59209c95-0da3-458c-bdf5-9dc61588d47d admin admin] PATH_INFO: `/v3/projects/76ef47dc1d6f4d1daf05f9992c784414/users/e264caf4e9d34cc58ed71d94d090adbf/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.593102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59209c95-0da3-458c-bdf5-9dc61588d47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ef47dc1d6f4d1daf05f9992c784414, user_id=e264caf4e9d34cc58ed71d94d090adbf, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.595779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59209c95-0da3-458c-bdf5-9dc61588d47d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.628251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] The token cache is being invalidate because project c94ae79784be4fddbd4b85933d4bcf5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:15.628543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:15.629193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9e8788ea-4b1d-4913-85ab-1afe1a126281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:15.634969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1340/2643] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:15 2022] DELETE /identity/v3/projects/c94ae79784be4fddbd4b85933d4bcf5b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:15.636518 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8db1a-7d59-4ce4-9923-6e3759cbc737 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.636912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8db1a-7d59-4ce4-9923-6e3759cbc737 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.637301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8db1a-7d59-4ce4-9923-6e3759cbc737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.657248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-21f8db1a-7d59-4ce4-9923-6e3759cbc737 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:15.658460 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-21f8db1a-7d59-4ce4-9923-6e3759cbc737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:15.666555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1341/2644] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:15.678706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d868b-a294-4a84-95a2-ce6c920e0029 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.679606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d868b-a294-4a84-95a2-ce6c920e0029 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.679985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d868b-a294-4a84-95a2-ce6c920e0029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.698639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-974d868b-a294-4a84-95a2-ce6c920e0029 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:15.698892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1304/2645] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:15 2022] PUT /identity/v3/projects/76ef47dc1d6f4d1daf05f9992c784414/users/e264caf4e9d34cc58ed71d94d090adbf/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:15.701023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-974d868b-a294-4a84-95a2-ce6c920e0029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:15.711350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.711648 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.711956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.739408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] MFA Rules not processed for user `e264caf4e9d34cc58ed71d94d090adbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:15.740950 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:15.774150 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1342/2646] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:15.778206 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d868b-a294-4a84-95a2-ce6c920e0029 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.793856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d868b-a294-4a84-95a2-ce6c920e0029 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.793856 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.794536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.794911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.795190 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.795978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:15.796284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.797916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.810547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1343/2647] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 15:22:15.814153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.814941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1305/2648] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:15.824063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b4b1b-fd1c-44da-b64f-05fafefee9a0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.826524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.827981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:15.828837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.829487 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.829768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.830039 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.830719 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.832678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.884696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd45d8b-435d-4a7c-af17-576f3f8d6cac None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:15.888356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:15.889005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:15.889291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:15.889570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:15.898592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:15.900189 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:15.908806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1306/2649] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:15.942380 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1344/2650] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:15.958537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:15.969519 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab1bd7-5e50-4158-8362-5880a7aeaf33 service glance] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.032877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.033181 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.033548 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.033940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.040977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.042937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.143676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1307/2651] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:15 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:16.145356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:16.158882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be4eb4-8913-4fa2-8339-4b701ae452a8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.165576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.169928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.169928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.169928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.169928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.169928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.188578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1345/2652] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:16.193041 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:16.208446 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3fb1-28b4-4ab2-b643-d7e706bdbba6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.209771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18dcb8-e211-4d8a-b7bd-682991a6cece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.210822 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18dcb8-e211-4d8a-b7bd-682991a6cece admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.211403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18dcb8-e211-4d8a-b7bd-682991a6cece admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.212026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18dcb8-e211-4d8a-b7bd-682991a6cece admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.213147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18dcb8-e211-4d8a-b7bd-682991a6cece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.214752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18dcb8-e211-4d8a-b7bd-682991a6cece admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.225462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1308/2653] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:16.239885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:16.247098 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a0629-f739-4805-81b4-588ddd6daf72 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.252198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.253102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.253382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.253658 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] PATH_INFO: `/v3/projects/c477057f8b8849f8ab3f6924bc36f679/users/ce8638a8d33c4a74ab06ff358e8d49a4/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.269705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c477057f8b8849f8ab3f6924bc36f679, user_id=ce8638a8d33c4a74ab06ff358e8d49a4, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.272718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.360533 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1346/2654] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:16 2022] PUT /identity/v3/projects/c477057f8b8849f8ab3f6924bc36f679/users/ce8638a8d33c4a74ab06ff358e8d49a4/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:16.372884 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8495c-661a-48bc-a7ce-be6d2b028a6f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.373172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8495c-661a-48bc-a7ce-be6d2b028a6f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.373452 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8495c-661a-48bc-a7ce-be6d2b028a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.400667 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-66b8495c-661a-48bc-a7ce-be6d2b028a6f None None] MFA Rules not processed for user `ce8638a8d33c4a74ab06ff358e8d49a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:16.401877 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-66b8495c-661a-48bc-a7ce-be6d2b028a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:16.464637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1309/2655] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:16.469136 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:16.534067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49875f4-a8c4-405b-b784-cd26b43d983e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.539138 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.539831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.541181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.541181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.541181 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.543074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.644778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1347/2656] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:16 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:16.645830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8495c-661a-48bc-a7ce-be6d2b028a6f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:16.657552 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8495c-661a-48bc-a7ce-be6d2b028a6f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.716492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.717187 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.717508 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.717818 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.718583 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.720609 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.797521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1310/2657] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:16 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:16.801589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:16.813064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415c444-a002-48c4-9c67-46b790cf4549 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.818471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.819140 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.819423 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.819804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.820727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.822392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.844122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1348/2658] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:16.849296 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:16.860570 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad9b16-b7cc-4f67-a527-689418164803 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:16.865832 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:16.866722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:16.867030 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:16.868274 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] PATH_INFO: `/v3/projects/5a2a97d8d4b14af292cd51684e3ac562/users/a420a0d623cd463d97182a6796af4f25/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:16.894938 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2a97d8d4b14af292cd51684e3ac562, user_id=a420a0d623cd463d97182a6796af4f25, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:16.898346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:16.986133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1311/2659] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:16 2022] PUT /identity/v3/projects/5a2a97d8d4b14af292cd51684e3ac562/users/a420a0d623cd463d97182a6796af4f25/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:16.993684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:17.003578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e9eac-5036-4267-9127-b24fb500f7d0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:17.060269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ccc81-1461-45cc-9dbf-06dc17ad84bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:17.060930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ccc81-1461-45cc-9dbf-06dc17ad84bb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:17.061212 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ccc81-1461-45cc-9dbf-06dc17ad84bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:17.061489 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ccc81-1461-45cc-9dbf-06dc17ad84bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:17.062211 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ccc81-1461-45cc-9dbf-06dc17ad84bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:17.063841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ccc81-1461-45cc-9dbf-06dc17ad84bb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:17.073359 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1349/2660] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:17.077664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:17.088751 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a6540-a2e2-4d99-a51a-e1448cfcf003 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:17.093406 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:17.094285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:17.094574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:17.094850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] PATH_INFO: `/v3/projects/5a2a97d8d4b14af292cd51684e3ac562/users/a420a0d623cd463d97182a6796af4f25/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:17.096819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2a97d8d4b14af292cd51684e3ac562, user_id=a420a0d623cd463d97182a6796af4f25, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:17.099429 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:17.110940 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1312/2661] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:17 2022] PUT /identity/v3/projects/5a2a97d8d4b14af292cd51684e3ac562/users/a420a0d623cd463d97182a6796af4f25/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:17.121453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ad681-00ff-43ff-b018-8621b1374d5f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:17.121826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ad681-00ff-43ff-b018-8621b1374d5f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:17.122108 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ad681-00ff-43ff-b018-8621b1374d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:17.149146 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4d3ad681-00ff-43ff-b018-8621b1374d5f None None] MFA Rules not processed for user `a420a0d623cd463d97182a6796af4f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:17.150331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4d3ad681-00ff-43ff-b018-8621b1374d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:17.216024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1350/2662] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:17.227904 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:17.229862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:17.288083 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cedc1e-d65c-4bd6-88c4-8f2f820e54bc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:17.292514 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f0ce9-d85e-4d45-813c-41e29e251ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:17.293162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f0ce9-d85e-4d45-813c-41e29e251ccb service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:17.293442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f0ce9-d85e-4d45-813c-41e29e251ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:17.293727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f0ce9-d85e-4d45-813c-41e29e251ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:17.302947 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f0ce9-d85e-4d45-813c-41e29e251ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:17.304555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f0ce9-d85e-4d45-813c-41e29e251ccb service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:17.323801 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1313/2663] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:18.412678 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ad681-00ff-43ff-b018-8621b1374d5f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:18.489728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ad681-00ff-43ff-b018-8621b1374d5f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:18.496648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:18.497543 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:18.497851 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:18.498159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] PATH_INFO: `/v3/users/ce8638a8d33c4a74ab06ff358e8d49a4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:18.499443 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8638a8d33c4a74ab06ff358e8d49a4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:18.501438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:18.667021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8638a8d33c4a74ab06ff358e8d49a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:18.748449 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8638a8d33c4a74ab06ff358e8d49a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:18.755969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8638a8d33c4a74ab06ff358e8d49a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:18.756661 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8638a8d33c4a74ab06ff358e8d49a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:18.769539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8638a8d33c4a74ab06ff358e8d49a4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:18.775979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1351/2664] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:18 2022] DELETE /identity/v3/users/ce8638a8d33c4a74ab06ff358e8d49a4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:18.780439 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:18.780897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:18.780992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:18.798226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:18.799668 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:18.860135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1314/2665] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:18.871132 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:18.872584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:18.936214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548b91e-244b-4537-8a2e-85a08ece159e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:18.939998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:18.940651 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:18.940936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:18.941213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:18.951335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:18.952740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:18.963156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1352/2666] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:19.370858 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:19.372411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:19.384616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abb963-4e31-4a8f-9342-46917154d814 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:19.389522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:19.392559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:19.393197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:19.393493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] PATH_INFO: `/v3/projects/c477057f8b8849f8ab3f6924bc36f679` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:19.394771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c477057f8b8849f8ab3f6924bc36f679)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:19.397108 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:19.505791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c477057f8b8849f8ab3f6924bc36f679', 'request_id': 'req-c3e80b25-8444-4af8-b693-34a33e9c57a3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.622907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c477057f8b8849f8ab3f6924bc36f679', 'request_id': 'req-c3e80b25-8444-4af8-b693-34a33e9c57a3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.628833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c477057f8b8849f8ab3f6924bc36f679', 'request_id': 'req-c3e80b25-8444-4af8-b693-34a33e9c57a3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.640515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] The token cache is being invalidate because project c477057f8b8849f8ab3f6924bc36f679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:19.640798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.641425 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.643455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1315/2667] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:19 2022] DELETE /identity/v3/projects/c477057f8b8849f8ab3f6924bc36f679 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:19.647386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:19.658000 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f15f-1e6c-4130-b35f-1171ea5c84ad service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:19.716007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:19.716940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:19.717273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:19.717501 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] PATH_INFO: `/v3/users/a420a0d623cd463d97182a6796af4f25` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:19.718735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a420a0d623cd463d97182a6796af4f25)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:19.720706 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:19.905796 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a420a0d623cd463d97182a6796af4f25'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.968562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a420a0d623cd463d97182a6796af4f25'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.976103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a420a0d623cd463d97182a6796af4f25'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.979498 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a420a0d623cd463d97182a6796af4f25'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:19.989182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bab2ab65-790a-46e9-b2ce-acace9e8587f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a420a0d623cd463d97182a6796af4f25'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:20.003531 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1353/2668] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:19 2022] DELETE /identity/v3/users/a420a0d623cd463d97182a6796af4f25 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:20.448704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:20.450228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:20.464329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80b25-8444-4af8-b693-34a33e9c57a3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:20.531498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:20.532462 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:20.532815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:20.533094 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] PATH_INFO: `/v3/projects/5a2a97d8d4b14af292cd51684e3ac562` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:20.534723 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a2a97d8d4b14af292cd51684e3ac562)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:20.536760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:20.654920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2a97d8d4b14af292cd51684e3ac562', 'request_id': 'req-f63a157c-8441-4dde-a64e-44792f119a6b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:20.748917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2a97d8d4b14af292cd51684e3ac562', 'request_id': 'req-f63a157c-8441-4dde-a64e-44792f119a6b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:20.754958 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2a97d8d4b14af292cd51684e3ac562', 'request_id': 'req-f63a157c-8441-4dde-a64e-44792f119a6b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:20.766456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] The token cache is being invalidate because project 5a2a97d8d4b14af292cd51684e3ac562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:20.766766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:20.767413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f63a157c-8441-4dde-a64e-44792f119a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:20.768983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1316/2669] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:20 2022] DELETE /identity/v3/projects/5a2a97d8d4b14af292cd51684e3ac562 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:20.774726 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac943cd-948e-4c5d-9668-198a5f3610ca None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:20.775232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac943cd-948e-4c5d-9668-198a5f3610ca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:20.775437 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac943cd-948e-4c5d-9668-198a5f3610ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:20.800749 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4ac943cd-948e-4c5d-9668-198a5f3610ca None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:20.801985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4ac943cd-948e-4c5d-9668-198a5f3610ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:20.862982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1354/2670] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:20.871171 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae4c9e-8de9-4dbe-a518-f2bd422e1276 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:20.871463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae4c9e-8de9-4dbe-a518-f2bd422e1276 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:20.871776 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae4c9e-8de9-4dbe-a518-f2bd422e1276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:20.895087 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a3ae4c9e-8de9-4dbe-a518-f2bd422e1276 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:20.896618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a3ae4c9e-8de9-4dbe-a518-f2bd422e1276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:20.905324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1317/2671] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:20.909590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac943cd-948e-4c5d-9668-198a5f3610ca None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:20.921387 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac943cd-948e-4c5d-9668-198a5f3610ca None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:20.929137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:20.930515 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:20.930874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:20.931152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:20.931949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:22:20.932253 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:20.933874 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:20.948127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1355/2672] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:22:20.952159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae4c9e-8de9-4dbe-a518-f2bd422e1276 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:20.961139 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae4c9e-8de9-4dbe-a518-f2bd422e1276 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:20.965850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:20.966498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:20.966780 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:20.967050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:20.967767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:20.969653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:21.086067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1318/2673] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:22:20 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:22:21.088470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:21.098573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f40239-bb9b-4474-8cd2-ba1e711a1a31 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:21.157290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:21.158269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:21.158555 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:21.158824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:21.159511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:21.161642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:21.249144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1356/2674] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:22:21 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:22:21.253665 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:21.264308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0c18-d818-453e-9c10-4aaab32ce185 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:21.269228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:21.269939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:21.270217 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:21.270487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:21.271326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:21.273202 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:21.283740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1319/2675] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:21.287830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:21.298203 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75409e8-6305-4433-9826-c1cfbfdeb969 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:21.302784 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb88fc-c33f-4b90-8946-7b44198a2639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:21.303559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb88fc-c33f-4b90-8946-7b44198a2639 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:21.303878 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb88fc-c33f-4b90-8946-7b44198a2639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:21.304159 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb88fc-c33f-4b90-8946-7b44198a2639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:21.305675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb88fc-c33f-4b90-8946-7b44198a2639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:21.307725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb88fc-c33f-4b90-8946-7b44198a2639 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:21.319218 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1357/2676] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:22:21.325572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:21.335847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac5bb6-490a-4fa1-9393-6f625f1b3262 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:21.341131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:21.341999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:21.342304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:21.342579 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] PATH_INFO: `/v3/projects/797cc8faa2204b1287cbcac0cd3d41f1/users/724e8eceb4b44d29a0f535515a042e80/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:21.357050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=797cc8faa2204b1287cbcac0cd3d41f1, user_id=724e8eceb4b44d29a0f535515a042e80, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:21.359735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:21.415134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1320/2677] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:22:21 2022] PUT /identity/v3/projects/797cc8faa2204b1287cbcac0cd3d41f1/users/724e8eceb4b44d29a0f535515a042e80/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:21.420229 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009511c4-9953-4d2b-ba0d-7c948e588ae1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:21.420527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009511c4-9953-4d2b-ba0d-7c948e588ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:21.420824 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009511c4-9953-4d2b-ba0d-7c948e588ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:21.439011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-009511c4-9953-4d2b-ba0d-7c948e588ae1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:21.440399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-009511c4-9953-4d2b-ba0d-7c948e588ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:21.503313 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1358/2678] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:21.513653 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:21.515468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:21.580089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96a1a1-00db-49b6-a9ac-33e2bd0fce29 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:21.583725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:21.584399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:21.584683 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:21.584960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:21.595753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:21.597195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:21.608487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1321/2679] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:22.934718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1386b74-e109-48a1-a75a-22949312b735 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:22.935768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1386b74-e109-48a1-a75a-22949312b735 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:22.936064 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1386b74-e109-48a1-a75a-22949312b735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:22.979131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b1386b74-e109-48a1-a75a-22949312b735 None None] MFA Rules not processed for user `724e8eceb4b44d29a0f535515a042e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:22.980476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b1386b74-e109-48a1-a75a-22949312b735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:23.102882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1359/2680] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:23.115615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:23.123358 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:23.186003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c9eb-7c38-49f1-8d5f-7d887a1f3840 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:23.194376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:23.195133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:23.196063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:23.196738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:23.210101 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:23.211617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:23.223750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1322/2681] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:24.648163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86a40a-c59f-4b93-b1e5-cc84e1a3b69c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:24.648631 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86a40a-c59f-4b93-b1e5-cc84e1a3b69c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:24.648778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86a40a-c59f-4b93-b1e5-cc84e1a3b69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:24.671607 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fe86a40a-c59f-4b93-b1e5-cc84e1a3b69c None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:24.672871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fe86a40a-c59f-4b93-b1e5-cc84e1a3b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:24.726817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1360/2682] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:22:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:24.878213 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:24.879644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:24.960235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f975f7b-338c-4b8b-8eea-66bc5a885efc service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:24.965752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:24.966421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:24.966702 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:24.966973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:24.974886 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:24.976513 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:24.984456 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1323/2683] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:22:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:22:25.130174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7968a-a32c-4fab-81f4-10e48f37e2e8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:25.130462 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7968a-a32c-4fab-81f4-10e48f37e2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:25.131023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7968a-a32c-4fab-81f4-10e48f37e2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:25.150426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dcc7968a-a32c-4fab-81f4-10e48f37e2e8 None None] MFA Rules not processed for user `724e8eceb4b44d29a0f535515a042e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:25.150724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dcc7968a-a32c-4fab-81f4-10e48f37e2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:25.157765 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1361/2684] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:25.486402 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:25.499019 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b20a-a872-4e88-b748-2a35974c4c81 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:25.502977 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:25.503674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:25.503974 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:25.504227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:25.506500 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:25.518626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:25.520851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:25.531671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1324/2685] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:25.904968 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907183-d1aa-45de-b714-85512da01da6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:25.905416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907183-d1aa-45de-b714-85512da01da6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:25.905506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907183-d1aa-45de-b714-85512da01da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:25.926110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-61907183-d1aa-45de-b714-85512da01da6 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:25.926110 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-61907183-d1aa-45de-b714-85512da01da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:25.932288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1362/2686] 10.222.0.71 () {58 vars in 967 bytes} [Mon May 16 15:22:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:25.944625 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:25.957344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846f53-a9b5-434e-bcee-e10fbe14fbb7 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:25.967197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:25.967197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:25.967197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:25.967197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:25.974204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:25.975865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:25.984858 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1325/2687] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:22:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:22:26.627379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724f350-e6e4-46c1-8547-6530076bdee2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:26.627764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724f350-e6e4-46c1-8547-6530076bdee2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:26.628116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724f350-e6e4-46c1-8547-6530076bdee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:26.649649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7724f350-e6e4-46c1-8547-6530076bdee2 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:26.651728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7724f350-e6e4-46c1-8547-6530076bdee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:26.659788 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1363/2688] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:22:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:26.674226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:26.684534 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c8da-fb4e-46c5-bc98-ca959139ca12 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:26.689522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:26.690280 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:26.690633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:26.691096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:26.700044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:26.701633 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:26.710100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1326/2689] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:22:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:22:27.283774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:27.284328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:27.284328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:27.303063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:27.304426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:27.312586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1364/2690] 10.222.0.71 () {58 vars in 967 bytes} [Mon May 16 15:22:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:27.333040 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:27.344355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93323-649c-4d6d-a1d2-ed1157da402d service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:27.350023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd6486-efe5-42ae-833d-f952f62ef0da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:27.359195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6486-efe5-42ae-833d-f952f62ef0da service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:27.359195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6486-efe5-42ae-833d-f952f62ef0da service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:27.359195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6486-efe5-42ae-833d-f952f62ef0da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:27.360750 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd6486-efe5-42ae-833d-f952f62ef0da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:27.362318 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd6486-efe5-42ae-833d-f952f62ef0da service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:27.370988 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1327/2691] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:22:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:22:30.111653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:30.116020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:30.128734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142afd4-9dd7-4cdf-9ae7-2e852c2f6ff4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:30.137689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:30.138569 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:30.138893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:30.139157 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] PATH_INFO: `/v3/users/d0154612674d46c8ae3f75ff3aef4a00` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:30.140392 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0154612674d46c8ae3f75ff3aef4a00)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:30.144438 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:30.330238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0154612674d46c8ae3f75ff3aef4a00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:30.402767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0154612674d46c8ae3f75ff3aef4a00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:30.410153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0154612674d46c8ae3f75ff3aef4a00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:30.410817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0154612674d46c8ae3f75ff3aef4a00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:30.423145 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0154612674d46c8ae3f75ff3aef4a00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:30.429258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1365/2692] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:30 2022] DELETE /identity/v3/users/d0154612674d46c8ae3f75ff3aef4a00 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:30.433806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:30.434286 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:30.434403 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:30.452337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:30.453545 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:30.512479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1328/2693] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:22:30.527597 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:30.528945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:30.591761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe36326-53e3-4001-9e3f-c02a95a6764a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:30.595998 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a4148-224f-447a-bd9e-bbe2cf2493d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:30.596650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a4148-224f-447a-bd9e-bbe2cf2493d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:30.596929 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a4148-224f-447a-bd9e-bbe2cf2493d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:30.597335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a4148-224f-447a-bd9e-bbe2cf2493d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:30.606611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a4148-224f-447a-bd9e-bbe2cf2493d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:30.608298 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a4148-224f-447a-bd9e-bbe2cf2493d5 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:30.618956 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1366/2694] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:22:30.847150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:30.849963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:30.862810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7e03-c1fd-49b6-8f9c-019e8db631a5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:30.868399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:30.869314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:30.869782 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:30.870063 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] PATH_INFO: `/v3/projects/8141ea82ed35431693530da3cf074b46` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:30.874473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8141ea82ed35431693530da3cf074b46)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:30.876615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:31.007861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8141ea82ed35431693530da3cf074b46', 'request_id': 'req-c18fbb8c-2669-4d40-b244-a71ec8982f4b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:31.091175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8141ea82ed35431693530da3cf074b46', 'request_id': 'req-c18fbb8c-2669-4d40-b244-a71ec8982f4b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:31.098045 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8141ea82ed35431693530da3cf074b46', 'request_id': 'req-c18fbb8c-2669-4d40-b244-a71ec8982f4b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:31.109847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] The token cache is being invalidate because project 8141ea82ed35431693530da3cf074b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:31.110130 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:31.110773 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:31.112441 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1329/2695] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:30 2022] DELETE /identity/v3/projects/8141ea82ed35431693530da3cf074b46 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:49.717648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e67d-2a88-4b4c-a538-0d274c688f5b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:49.718172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e67d-2a88-4b4c-a538-0d274c688f5b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:49.718286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e67d-2a88-4b4c-a538-0d274c688f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:49.736566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e4c3e67d-2a88-4b4c-a538-0d274c688f5b None None] MFA Rules not processed for user `724e8eceb4b44d29a0f535515a042e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:49.737790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e4c3e67d-2a88-4b4c-a538-0d274c688f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:49.794004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1367/2696] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:58.689738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:58.691165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:58.758506 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fbb8c-2669-4d40-b244-a71ec8982f4b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:58.763230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:58.764121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:58.764404 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:58.764674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] PATH_INFO: `/v3/users/e264caf4e9d34cc58ed71d94d090adbf` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:58.765846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e264caf4e9d34cc58ed71d94d090adbf)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:58.767799 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:58.967736 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e264caf4e9d34cc58ed71d94d090adbf'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.039849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e264caf4e9d34cc58ed71d94d090adbf'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.040483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e264caf4e9d34cc58ed71d94d090adbf'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.053177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e264caf4e9d34cc58ed71d94d090adbf'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.060103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e264caf4e9d34cc58ed71d94d090adbf'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.065897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1330/2697] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:22:58 2022] DELETE /identity/v3/users/e264caf4e9d34cc58ed71d94d090adbf => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:22:59.070510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:59.070963 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:59.071078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:59.088179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:59.089911 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:59.147904 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1368/2698] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:59.158311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:59.159759 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:59.215820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817f165-3575-451f-abce-79b689e70cf7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:59.219487 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cec78-239c-4d78-8bae-6be5332d1551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:59.220100 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cec78-239c-4d78-8bae-6be5332d1551 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:59.220382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cec78-239c-4d78-8bae-6be5332d1551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:59.220649 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cec78-239c-4d78-8bae-6be5332d1551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:59.229426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cec78-239c-4d78-8bae-6be5332d1551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:59.231464 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cec78-239c-4d78-8bae-6be5332d1551 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:59.241396 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1331/2699] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:22:59.650024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:22:59.652608 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:59.666702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7b1b5-86f2-4a62-8480-3d45a5092fc7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:22:59.672114 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:22:59.672988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:59.673265 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:59.673535 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] PATH_INFO: `/v3/projects/76ef47dc1d6f4d1daf05f9992c784414` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:59.674718 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ef47dc1d6f4d1daf05f9992c784414)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:22:59.676792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:22:59.797728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ef47dc1d6f4d1daf05f9992c784414', 'request_id': 'req-e5fee35b-9039-466a-baac-23ddcd64b10f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.802988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ef47dc1d6f4d1daf05f9992c784414', 'request_id': 'req-e5fee35b-9039-466a-baac-23ddcd64b10f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.809654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ef47dc1d6f4d1daf05f9992c784414', 'request_id': 'req-e5fee35b-9039-466a-baac-23ddcd64b10f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.879804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] The token cache is being invalidate because project 76ef47dc1d6f4d1daf05f9992c784414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:22:59.880113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.880735 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5fee35b-9039-466a-baac-23ddcd64b10f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:22:59.882404 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1369/2700] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:22:59 2022] DELETE /identity/v3/projects/76ef47dc1d6f4d1daf05f9992c784414 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:22:59.888444 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59afed1-ef9a-44fd-9491-76f6df0f2538 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:59.888892 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59afed1-ef9a-44fd-9491-76f6df0f2538 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:59.889005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59afed1-ef9a-44fd-9491-76f6df0f2538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:59.907614 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f59afed1-ef9a-44fd-9491-76f6df0f2538 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:59.908847 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f59afed1-ef9a-44fd-9491-76f6df0f2538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:22:59.969474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1332/2701] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:22:59.977217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09fb50-248e-42dd-9603-e39b69f6e51c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:22:59.977442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09fb50-248e-42dd-9603-e39b69f6e51c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:22:59.977725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09fb50-248e-42dd-9603-e39b69f6e51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:22:59.997216 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3d09fb50-248e-42dd-9603-e39b69f6e51c None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:22:59.998230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3d09fb50-248e-42dd-9603-e39b69f6e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:00.006308 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1370/2702] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:00.010673 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59afed1-ef9a-44fd-9491-76f6df0f2538 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:00.021209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59afed1-ef9a-44fd-9491-76f6df0f2538 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:00.026496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:00.027333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.027745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.028029 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.028769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:23:00.029079 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:00.030680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:00.042242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1333/2703] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:23:00.046798 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09fb50-248e-42dd-9603-e39b69f6e51c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:00.056166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09fb50-248e-42dd-9603-e39b69f6e51c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:00.065527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:00.066248 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.066529 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.066807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.067511 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:00.069611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:00.160767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1371/2704] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:23:00 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:23:00.165124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:00.175204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4e028-fada-4bf0-a146-eb812d5a5600 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:00.228908 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:00.229494 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.229788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.230603 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.231265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:00.233210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:00.389410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1334/2705] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:23:00 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:23:00.394830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:00.405959 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab2828-373b-453f-b5f1-9f38fb5e674a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:00.410429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:00.411055 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.411336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.411611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.412388 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:00.413978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:00.425274 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1372/2706] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:23:00.429052 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:00.439479 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3e9ea-92dd-4341-9099-c11bedb1e71a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:00.445483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8031-8c2f-4ace-934c-f50ac7ca5c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:00.445483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8031-8c2f-4ace-934c-f50ac7ca5c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.445483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8031-8c2f-4ace-934c-f50ac7ca5c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.445483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8031-8c2f-4ace-934c-f50ac7ca5c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.446010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a8031-8c2f-4ace-934c-f50ac7ca5c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:00.447585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a8031-8c2f-4ace-934c-f50ac7ca5c06 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:00.457404 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1335/2707] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:00.461615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:00.473230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a8b94-90c0-4cd2-baa9-8d3d128b8e60 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:00.478022 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:00.486071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.486071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.486071 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] PATH_INFO: `/v3/projects/c2bec5dc47d64eb69425e31fbf83fd0a/users/e8d1748b290645d6bca294da82698d70/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.496371 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bec5dc47d64eb69425e31fbf83fd0a, user_id=e8d1748b290645d6bca294da82698d70, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:00.499309 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:00.625999 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1373/2708] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:23:00 2022] PUT /identity/v3/projects/c2bec5dc47d64eb69425e31fbf83fd0a/users/e8d1748b290645d6bca294da82698d70/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:00.636957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.637220 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.637499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.664556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] MFA Rules not processed for user `e8d1748b290645d6bca294da82698d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:00.665764 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:00.735819 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1336/2709] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:00.742389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:00.746411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:00.809732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa140e3-99ec-42bc-a4b3-389d4b2737d4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:00.815507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb3726-40ab-4d3e-b3a7-20b2d847bb0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:00.816258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb3726-40ab-4d3e-b3a7-20b2d847bb0d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:00.816872 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb3726-40ab-4d3e-b3a7-20b2d847bb0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:00.817465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb3726-40ab-4d3e-b3a7-20b2d847bb0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:00.830008 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb3726-40ab-4d3e-b3a7-20b2d847bb0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:00.831609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb3726-40ab-4d3e-b3a7-20b2d847bb0d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:00.843601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1374/2710] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:23:01.490481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:01.491973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:01.556427 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b455b1a-6b77-4047-9c3f-b27c20847c04 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:01.561490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:01.562354 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:01.565291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:01.565291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] PATH_INFO: `/v3/users/ba4d468428d24fc4be20c8583692794a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:01.565291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4d468428d24fc4be20c8583692794a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:01.566008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:01.822198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4d468428d24fc4be20c8583692794a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:01.934798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4d468428d24fc4be20c8583692794a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:01.935447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4d468428d24fc4be20c8583692794a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:01.947101 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4d468428d24fc4be20c8583692794a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:01.954195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4d468428d24fc4be20c8583692794a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:01.960252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1337/2711] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:23:01 2022] DELETE /identity/v3/users/ba4d468428d24fc4be20c8583692794a => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:01.965377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:01.965935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:01.965935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:01.982797 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:01.984017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:02.039049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1375/2712] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:02.049216 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:02.050617 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:02.108640 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95fcdd-0040-47f4-8c68-ed85ef86a0f2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:02.112215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:02.112883 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:02.113167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:02.113443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:02.123789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:02.125255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:02.134395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1338/2713] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:02.566092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:02.568525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:02.584390 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe7250-ed28-49e1-9026-189155cae904 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:02.597461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:02.597461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:02.597461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:02.597461 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] PATH_INFO: `/v3/projects/f8110316a04949afbb214f4f46f34c77` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:02.598330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8110316a04949afbb214f4f46f34c77)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:02.600416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:02.748776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8110316a04949afbb214f4f46f34c77', 'request_id': 'req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:02.753973 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8110316a04949afbb214f4f46f34c77', 'request_id': 'req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:02.764721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8110316a04949afbb214f4f46f34c77', 'request_id': 'req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:02.834004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] The token cache is being invalidate because project f8110316a04949afbb214f4f46f34c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:23:02.834224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:02.834930 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:02.836539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1376/2714] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:23:02 2022] DELETE /identity/v3/projects/f8110316a04949afbb214f4f46f34c77 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:02.841728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:02.851504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5c7b-7bc1-4aff-9faa-11c94fbb9a14 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:02.909499 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:02.910376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:02.910660 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:02.910937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] PATH_INFO: `/v3/users/fde7557d9b5d4abc9f33a7e39287a4f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:02.912259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde7557d9b5d4abc9f33a7e39287a4f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:02.914262 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:03.081804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde7557d9b5d4abc9f33a7e39287a4f4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.168257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde7557d9b5d4abc9f33a7e39287a4f4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.169961 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde7557d9b5d4abc9f33a7e39287a4f4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.181715 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde7557d9b5d4abc9f33a7e39287a4f4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.192304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34e126e-80d2-4022-8b45-1e7760a0575c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde7557d9b5d4abc9f33a7e39287a4f4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.198062 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1339/2715] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:23:02 2022] DELETE /identity/v3/users/fde7557d9b5d4abc9f33a7e39287a4f4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:03.666250 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:03.668090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:03.684103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba8ca1-dfe1-43cc-9be3-6dd855fd930e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:03.750353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:03.751213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:03.751488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:03.751786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] PATH_INFO: `/v3/projects/9522eb9c52184a39a8deb29ec4141ede` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:03.753069 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9522eb9c52184a39a8deb29ec4141ede)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:03.755141 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:03.871343 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9522eb9c52184a39a8deb29ec4141ede', 'request_id': 'req-2760357f-7356-42a6-9ae6-24e703ae3bb6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.877043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9522eb9c52184a39a8deb29ec4141ede', 'request_id': 'req-2760357f-7356-42a6-9ae6-24e703ae3bb6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.883273 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9522eb9c52184a39a8deb29ec4141ede', 'request_id': 'req-2760357f-7356-42a6-9ae6-24e703ae3bb6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.959154 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] The token cache is being invalidate because project 9522eb9c52184a39a8deb29ec4141ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:23:03.961689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.961689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2760357f-7356-42a6-9ae6-24e703ae3bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:03.961689 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1377/2716] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:23:03 2022] DELETE /identity/v3/projects/9522eb9c52184a39a8deb29ec4141ede => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:03.967421 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6265fc-9933-4b5d-9e9e-d670c609cf06 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:03.967952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6265fc-9933-4b5d-9e9e-d670c609cf06 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:03.968058 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6265fc-9933-4b5d-9e9e-d670c609cf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:03.985732 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ba6265fc-9933-4b5d-9e9e-d670c609cf06 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:03.986911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ba6265fc-9933-4b5d-9e9e-d670c609cf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:04.044069 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1340/2717] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:04.051853 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f156d6-d442-4f40-93b8-7de828f37214 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.052143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f156d6-d442-4f40-93b8-7de828f37214 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.052421 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f156d6-d442-4f40-93b8-7de828f37214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.071116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-56f156d6-d442-4f40-93b8-7de828f37214 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:04.081838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-56f156d6-d442-4f40-93b8-7de828f37214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:04.081838 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1378/2718] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:04.084839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6265fc-9933-4b5d-9e9e-d670c609cf06 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.094897 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6265fc-9933-4b5d-9e9e-d670c609cf06 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.099379 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.100055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.100338 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.100610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.101324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:23:04.101626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.103193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:04.114779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1341/2719] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:23:04.118921 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f156d6-d442-4f40-93b8-7de828f37214 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.128105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f156d6-d442-4f40-93b8-7de828f37214 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.133766 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.133985 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.134277 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.134556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.135251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.137187 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:04.261422 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1379/2720] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:23:04 2022] POST /identity/v3/projects => generated 393 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:04.262478 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.273806 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ca043-1b62-4487-bf45-5fc5c3a7b9f8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.329400 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.330109 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.330394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.330674 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.331376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.333337 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:04.425197 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1342/2721] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:23:04 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:04.429347 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.440352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33d1d0-1e15-40c5-b1b8-39302481ea6c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.445183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.445836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.446122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.446399 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.447215 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.448970 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:04.463263 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1380/2722] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:04.464267 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.474758 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611ec30-3135-4c79-a453-4be1c8be627e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.480027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88772484-4935-48d4-b598-219e44fc4d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.480701 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772484-4935-48d4-b598-219e44fc4d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.480986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772484-4935-48d4-b598-219e44fc4d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.481264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772484-4935-48d4-b598-219e44fc4d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.482002 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88772484-4935-48d4-b598-219e44fc4d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.483745 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88772484-4935-48d4-b598-219e44fc4d80 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:04.494460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1343/2723] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:04.499609 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.510120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9eb57-f312-4514-8c89-6115f0a16401 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.515528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.516431 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.516709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.516983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] PATH_INFO: `/v3/projects/ef783dd02ddb445baa4f10974616e3d9/users/ec05e50a291b43258a3c7ee5789415b0/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.536143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef783dd02ddb445baa4f10974616e3d9, user_id=ec05e50a291b43258a3c7ee5789415b0, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.538804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:04.605492 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1381/2724] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:23:04 2022] PUT /identity/v3/projects/ef783dd02ddb445baa4f10974616e3d9/users/ec05e50a291b43258a3c7ee5789415b0/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:04.615969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896f78-96ac-4b64-9688-3c321ffa00f8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.620335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896f78-96ac-4b64-9688-3c321ffa00f8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.620335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896f78-96ac-4b64-9688-3c321ffa00f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.648443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7e896f78-96ac-4b64-9688-3c321ffa00f8 None None] MFA Rules not processed for user `ec05e50a291b43258a3c7ee5789415b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:04.648443 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7e896f78-96ac-4b64-9688-3c321ffa00f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:04.719287 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1344/2725] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:04.723909 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.785305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c0eca-2499-4c7b-8b01-a12765e81a96 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.789977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.790636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.790922 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.791202 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.791923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.793989 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:04.898573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1382/2726] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:23:04 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:04.907869 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e896f78-96ac-4b64-9688-3c321ffa00f8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:04.918956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e896f78-96ac-4b64-9688-3c321ffa00f8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:04.975547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:04.976255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:04.976442 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:04.976719 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:04.977411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:04.979356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:05.097700 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1345/2727] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:23:04 2022] POST /identity/v3/users => generated 302 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:05.098991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:05.109557 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1aa06c-ef37-4c86-8ec8-11588623d1fd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:05.114315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:05.115060 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:05.115415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:05.115846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:05.116630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:05.118306 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:05.130026 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1383/2728] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:05.134376 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:05.145000 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e380b3-15fb-49f5-bbc1-556396197307 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:05.149928 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:05.150791 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:05.151075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:05.151350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] PATH_INFO: `/v3/projects/3f3263c87dac4dab90275c3c8302efaa/users/b603335e65fe4e368d58ecf42b199126/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:05.166305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3263c87dac4dab90275c3c8302efaa, user_id=b603335e65fe4e368d58ecf42b199126, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:05.169133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:05.266218 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1346/2729] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:23:05 2022] PUT /identity/v3/projects/3f3263c87dac4dab90275c3c8302efaa/users/b603335e65fe4e368d58ecf42b199126/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:05.270072 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:05.280459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a103e9-89bb-4cd7-aa90-46886c9a1fc5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:05.336823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696f917-0c74-4177-a14e-3b45738b108b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:05.337488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696f917-0c74-4177-a14e-3b45738b108b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:05.337768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696f917-0c74-4177-a14e-3b45738b108b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:05.338083 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696f917-0c74-4177-a14e-3b45738b108b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:05.338801 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696f917-0c74-4177-a14e-3b45738b108b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:05.340434 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696f917-0c74-4177-a14e-3b45738b108b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:05.350630 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1384/2730] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:05.354837 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:05.365401 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922c1dd-37b6-4d1e-96b8-f7051880c0c6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:05.370003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:05.370878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:05.371157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:05.371431 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] PATH_INFO: `/v3/projects/3f3263c87dac4dab90275c3c8302efaa/users/b603335e65fe4e368d58ecf42b199126/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:05.374273 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3263c87dac4dab90275c3c8302efaa, user_id=b603335e65fe4e368d58ecf42b199126, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:05.378289 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:05.386911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1347/2731] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:23:05 2022] PUT /identity/v3/projects/3f3263c87dac4dab90275c3c8302efaa/users/b603335e65fe4e368d58ecf42b199126/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:05.398366 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:05.399016 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:05.399297 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:05.427284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] MFA Rules not processed for user `b603335e65fe4e368d58ecf42b199126`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:05.428528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:05.496096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1385/2732] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:05.507121 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:05.508865 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:05.567852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad60b3-86c5-4962-9d53-8325687ad122 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:05.571522 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:05.572195 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:05.572566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:05.572838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:05.581882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:05.583304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:05.592802 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1348/2733] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:05.643758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:05.645149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:05.657975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07e61e-9f0b-44b5-ae0e-4eedc067e62a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:05.661815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31503556-2662-4032-a3a6-3a9d63a593ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:05.662456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31503556-2662-4032-a3a6-3a9d63a593ca service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:05.662740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31503556-2662-4032-a3a6-3a9d63a593ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:05.663043 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31503556-2662-4032-a3a6-3a9d63a593ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:05.718566 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31503556-2662-4032-a3a6-3a9d63a593ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:05.720101 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31503556-2662-4032-a3a6-3a9d63a593ca service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:05.728984 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1386/2734] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:10.525515 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:10.584996 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfdd4a-09ec-4c22-ae76-9beba4d5ed7c service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:10.589453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:10.590311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:10.590589 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:10.590858 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] PATH_INFO: `/v3/users/ec05e50a291b43258a3c7ee5789415b0` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:10.592198 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec05e50a291b43258a3c7ee5789415b0)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:10.594092 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:10.761394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec05e50a291b43258a3c7ee5789415b0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:10.822448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec05e50a291b43258a3c7ee5789415b0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:10.823093 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec05e50a291b43258a3c7ee5789415b0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:10.834689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec05e50a291b43258a3c7ee5789415b0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:10.841223 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec05e50a291b43258a3c7ee5789415b0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:10.847447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1349/2735] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:23:10 2022] DELETE /identity/v3/users/ec05e50a291b43258a3c7ee5789415b0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:10.851915 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:10.852394 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:10.852528 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:10.869698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:10.870896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:10.930065 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1387/2736] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:10.940160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:10.941557 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:10.996288 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ea971-58c0-4669-9cb6-bf4177fb9da8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:11.000535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:11.001205 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:11.001491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:11.001768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:11.010625 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:11.012135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:11.022158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1350/2737] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:11.595439 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:11.596986 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:11.609305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4294a-b32b-4102-b6da-68a6a8c8dab1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:11.614196 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:11.615099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:11.615382 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:11.615686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] PATH_INFO: `/v3/projects/ef783dd02ddb445baa4f10974616e3d9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:11.616931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef783dd02ddb445baa4f10974616e3d9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:11.618918 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:11.714235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef783dd02ddb445baa4f10974616e3d9', 'request_id': 'req-58eac151-8376-44e0-964d-cc3204027385'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:11.720286 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef783dd02ddb445baa4f10974616e3d9', 'request_id': 'req-58eac151-8376-44e0-964d-cc3204027385'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:11.726861 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef783dd02ddb445baa4f10974616e3d9', 'request_id': 'req-58eac151-8376-44e0-964d-cc3204027385'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:11.793837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] The token cache is being invalidate because project ef783dd02ddb445baa4f10974616e3d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:23:11.794126 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:11.795328 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:11.797164 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1388/2738] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:23:11 2022] DELETE /identity/v3/projects/ef783dd02ddb445baa4f10974616e3d9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:23:11.801285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:11.811392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a498af-42eb-4841-8a7a-292e9bedb28e service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:11.865120 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:11.865990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:11.866474 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:11.866747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] PATH_INFO: `/v3/users/b603335e65fe4e368d58ecf42b199126` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:11.868104 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b603335e65fe4e368d58ecf42b199126)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:11.870037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:12.052798 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b603335e65fe4e368d58ecf42b199126'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.131065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b603335e65fe4e368d58ecf42b199126'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.131724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b603335e65fe4e368d58ecf42b199126'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.144305 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b603335e65fe4e368d58ecf42b199126'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.151170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-78a91b32-934e-4e56-9004-18320e00b600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b603335e65fe4e368d58ecf42b199126'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.157804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1351/2739] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:23:11 2022] DELETE /identity/v3/users/b603335e65fe4e368d58ecf42b199126 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:12.582352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:12.583823 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:12.596847 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eac151-8376-44e0-964d-cc3204027385 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:12.654723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:12.655589 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:12.655901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:12.656178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] PATH_INFO: `/v3/projects/3f3263c87dac4dab90275c3c8302efaa` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:12.657346 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3263c87dac4dab90275c3c8302efaa)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:12.659318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:12.760331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3263c87dac4dab90275c3c8302efaa', 'request_id': 'req-acb90b7f-79e6-483d-8d1a-672272835cee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.766341 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3263c87dac4dab90275c3c8302efaa', 'request_id': 'req-acb90b7f-79e6-483d-8d1a-672272835cee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.773018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3263c87dac4dab90275c3c8302efaa', 'request_id': 'req-acb90b7f-79e6-483d-8d1a-672272835cee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.845271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] The token cache is being invalidate because project 3f3263c87dac4dab90275c3c8302efaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:23:12.845480 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.846086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acb90b7f-79e6-483d-8d1a-672272835cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:12.847764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1389/2740] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:23:12 2022] DELETE /identity/v3/projects/3f3263c87dac4dab90275c3c8302efaa => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:12.852848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09260207-c75d-4229-bfc4-48bcd3df8611 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:12.853328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09260207-c75d-4229-bfc4-48bcd3df8611 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:12.853543 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09260207-c75d-4229-bfc4-48bcd3df8611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:12.873857 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-09260207-c75d-4229-bfc4-48bcd3df8611 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:12.875394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-09260207-c75d-4229-bfc4-48bcd3df8611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:12.936562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1352/2741] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:12.944451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535caca-ebd6-4e7b-97c8-576aba6c1532 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:12.944744 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535caca-ebd6-4e7b-97c8-576aba6c1532 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:12.945027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535caca-ebd6-4e7b-97c8-576aba6c1532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:12.963679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3535caca-ebd6-4e7b-97c8-576aba6c1532 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:12.968950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3535caca-ebd6-4e7b-97c8-576aba6c1532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:12.973993 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1390/2742] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:12.978203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09260207-c75d-4229-bfc4-48bcd3df8611 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:12.988129 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09260207-c75d-4229-bfc4-48bcd3df8611 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:12.992637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:12.997859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:12.997859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:12.997859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:12.997859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:23:12.997859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:12.997859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:13.007644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1353/2743] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:23:13.011871 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535caca-ebd6-4e7b-97c8-576aba6c1532 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:13.021807 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535caca-ebd6-4e7b-97c8-576aba6c1532 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:13.026444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:13.027081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:13.027360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:13.027659 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:13.028331 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:13.030960 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:13.132881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1391/2744] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:23:13 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:13.136771 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:13.147301 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780361f4-d16d-4a31-9d9a-4c98149e9c1d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:13.200021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:13.200676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:13.201561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:13.201909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:13.202496 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:13.204419 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:13.322388 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1354/2745] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:23:13 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:13.327020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:13.337386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bc096-1be3-4634-9f3a-8edf5bf6280c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:13.341988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:13.342635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:13.342923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:13.343204 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:13.343964 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:13.345600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:13.355714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1392/2746] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:13.359608 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:13.369470 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ecb74-0ced-4375-b10c-e7f6998dd50c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:13.375020 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61493ffe-d402-4559-a530-0f80eb62cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:13.380473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61493ffe-d402-4559-a530-0f80eb62cf6d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:13.380473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61493ffe-d402-4559-a530-0f80eb62cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:13.380473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61493ffe-d402-4559-a530-0f80eb62cf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:13.380473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61493ffe-d402-4559-a530-0f80eb62cf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:13.380473 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61493ffe-d402-4559-a530-0f80eb62cf6d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:13.389590 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1355/2747] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:13.393358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:13.403552 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e394b8-e6d9-46ca-bf97-4ab017ad6f54 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:13.408396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:13.409281 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:13.409559 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:13.409941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] PATH_INFO: `/v3/projects/5e343d0e69de4a2c956a7ebc9056b8d5/users/d37b3b7824b24b328138061ae7af38f8/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:13.424173 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e343d0e69de4a2c956a7ebc9056b8d5, user_id=d37b3b7824b24b328138061ae7af38f8, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:13.426859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:13.525948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1393/2748] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:23:13 2022] PUT /identity/v3/projects/5e343d0e69de4a2c956a7ebc9056b8d5/users/d37b3b7824b24b328138061ae7af38f8/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:13.531265 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074386f-7c3f-4f63-b432-b2afd0e662f8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:13.531558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074386f-7c3f-4f63-b432-b2afd0e662f8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:13.531878 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074386f-7c3f-4f63-b432-b2afd0e662f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:13.550135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c074386f-7c3f-4f63-b432-b2afd0e662f8 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:13.551326 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c074386f-7c3f-4f63-b432-b2afd0e662f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:13.609416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1356/2749] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:13.618974 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:13.620763 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:13.678945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383cb46-139d-4c8c-a685-d92e00a03437 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:13.682540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:13.683183 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:13.683466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:13.683778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:13.693791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:13.695228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:13.704846 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1394/2750] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:14.850752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51fbc-3bfb-457b-84e7-effc919da309 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:14.851236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51fbc-3bfb-457b-84e7-effc919da309 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:14.851332 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51fbc-3bfb-457b-84e7-effc919da309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:14.883278 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-44d51fbc-3bfb-457b-84e7-effc919da309 None None] MFA Rules not processed for user `d37b3b7824b24b328138061ae7af38f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:14.884594 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-44d51fbc-3bfb-457b-84e7-effc919da309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:14.947483 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1357/2751] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:14.958663 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:14.960104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:15.016626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcf43-f5b7-418d-8b64-01b74049e509 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:15.020152 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d466cd1-18ab-42e9-842a-d622e79fb837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:15.020794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466cd1-18ab-42e9-842a-d622e79fb837 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:15.021078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466cd1-18ab-42e9-842a-d622e79fb837 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:15.021354 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466cd1-18ab-42e9-842a-d622e79fb837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:15.029736 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466cd1-18ab-42e9-842a-d622e79fb837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:15.031172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466cd1-18ab-42e9-842a-d622e79fb837 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:15.039785 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1395/2752] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:16.725530 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359339d-00cb-4757-bd34-548d5d9de526 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:16.726141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359339d-00cb-4757-bd34-548d5d9de526 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:16.726141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359339d-00cb-4757-bd34-548d5d9de526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:16.729221 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:16.729610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:16.729836 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:16.754264 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9359339d-00cb-4757-bd34-548d5d9de526 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:16.755491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9359339d-00cb-4757-bd34-548d5d9de526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:16.756510 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] MFA Rules not processed for user `d37b3b7824b24b328138061ae7af38f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:16.757791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:16.766654 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1396/2753] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:16.826055 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1358/2754] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:16.833199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:16.834905 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:16.913738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126e139-ef27-4b9e-bc0d-1517eca6c315 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:16.918777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb763c-0a71-45c8-9739-08652803029d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:16.919432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb763c-0a71-45c8-9739-08652803029d service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:16.919772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb763c-0a71-45c8-9739-08652803029d service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:16.920053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb763c-0a71-45c8-9739-08652803029d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:16.928269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb763c-0a71-45c8-9739-08652803029d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:16.929885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb763c-0a71-45c8-9739-08652803029d service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:16.947778 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1397/2755] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:23:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:23:17.031859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359339d-00cb-4757-bd34-548d5d9de526 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:17.040979 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359339d-00cb-4757-bd34-548d5d9de526 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:17.044520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:17.045294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:17.045574 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:17.045845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:17.047555 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:17.057095 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:17.058517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:17.067199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1359/2756] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:39.678122 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:39.678674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:39.678674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:39.698199 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:39.699386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:39.707386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1398/2757] 10.222.0.71 () {58 vars in 967 bytes} [Mon May 16 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:39.719803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:39.733152 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83232e0-de7b-49e6-86a9-dee9fe7ccd26 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:39.738060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:39.738709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:39.738992 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:39.739268 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:39.747334 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:39.749518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:39.757699 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1360/2758] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:23:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:23:42.916271 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:42.917815 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:42.933113 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb248-03c6-4067-a472-f1af9e9ef271 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:42.938479 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:42.939363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:42.939664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:42.939944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] PATH_INFO: `/v3/users/724e8eceb4b44d29a0f535515a042e80` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:42.941149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724e8eceb4b44d29a0f535515a042e80)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:42.943078 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:43.165562 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724e8eceb4b44d29a0f535515a042e80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.242611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724e8eceb4b44d29a0f535515a042e80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.250657 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724e8eceb4b44d29a0f535515a042e80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.253514 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724e8eceb4b44d29a0f535515a042e80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.263655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c96d34f4-e88d-46b7-99c3-42f9c1168761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724e8eceb4b44d29a0f535515a042e80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.269620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1399/2759] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:23:42 2022] DELETE /identity/v3/users/724e8eceb4b44d29a0f535515a042e80 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:43.533730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:43.547747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4e688-d045-4747-967e-27c1ba3e4437 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:43.610227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:43.611209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:43.611489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:43.615005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] PATH_INFO: `/v3/projects/797cc8faa2204b1287cbcac0cd3d41f1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:43.615005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=797cc8faa2204b1287cbcac0cd3d41f1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:43.620330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:43.727447 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '797cc8faa2204b1287cbcac0cd3d41f1', 'request_id': 'req-dc601732-94ce-4612-b596-5754170bc20f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.811009 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '797cc8faa2204b1287cbcac0cd3d41f1', 'request_id': 'req-dc601732-94ce-4612-b596-5754170bc20f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.817697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '797cc8faa2204b1287cbcac0cd3d41f1', 'request_id': 'req-dc601732-94ce-4612-b596-5754170bc20f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.829959 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] The token cache is being invalidate because project 797cc8faa2204b1287cbcac0cd3d41f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:23:43.830355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.831054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-dc601732-94ce-4612-b596-5754170bc20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:43.832890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1361/2760] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:23:43 2022] DELETE /identity/v3/projects/797cc8faa2204b1287cbcac0cd3d41f1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:43.837782 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3878-0257-465b-864a-ee5f5d722ec2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:43.838074 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3878-0257-465b-864a-ee5f5d722ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:43.838351 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3878-0257-465b-864a-ee5f5d722ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:43.859541 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6cfb3878-0257-465b-864a-ee5f5d722ec2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:43.860812 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6cfb3878-0257-465b-864a-ee5f5d722ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:43.924742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1400/2761] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:43.932711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ba308-a7e8-4e5b-af89-5e3dd6ec373b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:43.933378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ba308-a7e8-4e5b-af89-5e3dd6ec373b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:43.933952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ba308-a7e8-4e5b-af89-5e3dd6ec373b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:43.953302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-494ba308-a7e8-4e5b-af89-5e3dd6ec373b None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:43.954637 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-494ba308-a7e8-4e5b-af89-5e3dd6ec373b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:43.971537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1362/2762] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 15:23:43.976048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb3878-0257-465b-864a-ee5f5d722ec2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:43.990168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb3878-0257-465b-864a-ee5f5d722ec2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:43.994642 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:43.995304 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:43.995584 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:43.995893 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:43.996643 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:23:43.996945 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:43.998527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:44.012330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1401/2763] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:23:44.016550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ba308-a7e8-4e5b-af89-5e3dd6ec373b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:44.025844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ba308-a7e8-4e5b-af89-5e3dd6ec373b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:44.031848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:44.031848 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:44.032066 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:44.032172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:44.032821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:44.034713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:44.110135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1363/2764] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:23:44 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:44.114357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:44.124587 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0be57-b5d5-40a3-9352-031342ca4ab2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:44.182509 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:44.183189 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:44.183476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:44.183800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:44.184539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:44.186714 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:44.263077 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1402/2765] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:23:44 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:44.268722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:44.279111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831979a8-40da-49c0-859b-8d6b78edf269 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:44.284761 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:44.285471 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:44.285752 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:44.286021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:44.286747 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:44.288394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:44.299686 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1364/2766] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:44.303732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:44.314228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da000b-5ade-4a8e-9eac-53bf7d4ff674 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:44.319293 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a2b3-6c9c-447a-ac5d-aa4431c03ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:44.320001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a2b3-6c9c-447a-ac5d-aa4431c03ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:44.320300 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a2b3-6c9c-447a-ac5d-aa4431c03ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:44.320577 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a2b3-6c9c-447a-ac5d-aa4431c03ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:44.321301 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4a2b3-6c9c-447a-ac5d-aa4431c03ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:44.323001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4a2b3-6c9c-447a-ac5d-aa4431c03ea8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:44.334011 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1403/2767] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:44.338168 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:44.349086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646b044-caa1-451d-8aec-c8dbca4c281e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:44.356905 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:44.357825 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:44.358105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:44.358566 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] PATH_INFO: `/v3/projects/9c06bd5ee6e64ed08b78f9f22727e7d5/users/ae7b0bf112754991b243bc4cc8598231/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:44.374244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c06bd5ee6e64ed08b78f9f22727e7d5, user_id=ae7b0bf112754991b243bc4cc8598231, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:44.376766 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:44.462277 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1365/2768] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:23:44 2022] PUT /identity/v3/projects/9c06bd5ee6e64ed08b78f9f22727e7d5/users/ae7b0bf112754991b243bc4cc8598231/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:44.472803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:44.473144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:44.473389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:44.501675 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] MFA Rules not processed for user `ae7b0bf112754991b243bc4cc8598231`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:44.502941 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:44.570099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1404/2769] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:44.582461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:44.584328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:44.642165 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5d298-86d6-4b85-b33e-489f55ba6e8d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:44.645899 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c737343-2602-49d0-a49e-0ea19aefe541 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:44.646554 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c737343-2602-49d0-a49e-0ea19aefe541 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:44.647134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c737343-2602-49d0-a49e-0ea19aefe541 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:44.647410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c737343-2602-49d0-a49e-0ea19aefe541 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:44.657031 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c737343-2602-49d0-a49e-0ea19aefe541 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:44.658576 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c737343-2602-49d0-a49e-0ea19aefe541 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:44.667196 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1366/2770] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:49.379843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:49.381317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:49.446288 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc59f7c-6203-4377-94eb-66792e7bea18 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:49.450940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:49.455003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:49.455003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:49.455003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] PATH_INFO: `/v3/users/e8d1748b290645d6bca294da82698d70` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:49.455003 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d1748b290645d6bca294da82698d70)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:49.455894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:49.638358 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d1748b290645d6bca294da82698d70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:49.722621 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d1748b290645d6bca294da82698d70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:49.729840 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d1748b290645d6bca294da82698d70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:49.730483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d1748b290645d6bca294da82698d70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:49.742605 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d1748b290645d6bca294da82698d70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:49.748727 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1405/2771] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:23:49 2022] DELETE /identity/v3/users/e8d1748b290645d6bca294da82698d70 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:49.753276 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:49.753767 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:49.753881 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:49.774193 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:49.775394 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:49.835124 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1367/2772] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:49.846156 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:49.849334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:49.910711 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149ad33-6d28-4b50-a217-f933f4dc5ffa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:49.914247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db986e04-6234-4573-b27d-a5db2c6667d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:49.914896 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986e04-6234-4573-b27d-a5db2c6667d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:49.915178 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986e04-6234-4573-b27d-a5db2c6667d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:49.915456 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986e04-6234-4573-b27d-a5db2c6667d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:49.924585 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db986e04-6234-4573-b27d-a5db2c6667d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:49.926062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db986e04-6234-4573-b27d-a5db2c6667d5 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:49.935361 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1406/2773] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:23:50.339266 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:50.340728 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:50.352866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a08fd7-0734-433f-8cf3-6bd91a3daa95 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:50.358922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:50.360253 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:50.360684 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:50.360990 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] PATH_INFO: `/v3/projects/c2bec5dc47d64eb69425e31fbf83fd0a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:50.362560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2bec5dc47d64eb69425e31fbf83fd0a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:50.364561 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:50.477836 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2bec5dc47d64eb69425e31fbf83fd0a', 'request_id': 'req-734fc9ec-1454-43db-9333-ccd98d5b6c29'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:50.563373 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2bec5dc47d64eb69425e31fbf83fd0a', 'request_id': 'req-734fc9ec-1454-43db-9333-ccd98d5b6c29'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:50.569529 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2bec5dc47d64eb69425e31fbf83fd0a', 'request_id': 'req-734fc9ec-1454-43db-9333-ccd98d5b6c29'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:50.581177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] The token cache is being invalidate because project c2bec5dc47d64eb69425e31fbf83fd0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:23:50.581493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:50.582144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-734fc9ec-1454-43db-9333-ccd98d5b6c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:23:50.583706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1368/2774] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:23:50 2022] DELETE /identity/v3/projects/c2bec5dc47d64eb69425e31fbf83fd0a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:50.588593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a59ed-6898-4f5f-a1c5-163a59cee20f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:50.589048 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a59ed-6898-4f5f-a1c5-163a59cee20f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:50.589143 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a59ed-6898-4f5f-a1c5-163a59cee20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:50.607103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-106a59ed-6898-4f5f-a1c5-163a59cee20f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:50.608401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-106a59ed-6898-4f5f-a1c5-163a59cee20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:50.674104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1407/2775] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:50.681991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5103d5-6095-4074-8ce9-0d2c7be940c4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:50.682279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5103d5-6095-4074-8ce9-0d2c7be940c4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:50.682559 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5103d5-6095-4074-8ce9-0d2c7be940c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:50.701016 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4c5103d5-6095-4074-8ce9-0d2c7be940c4 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:50.702201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4c5103d5-6095-4074-8ce9-0d2c7be940c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:50.711489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1369/2776] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:50.715540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a59ed-6898-4f5f-a1c5-163a59cee20f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:50.725497 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a59ed-6898-4f5f-a1c5-163a59cee20f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:50.730240 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:50.730837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:50.731144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:50.731425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:50.732329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:23:50.732639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:50.734237 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:50.746326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1408/2777] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:23:50.751435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5103d5-6095-4074-8ce9-0d2c7be940c4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:50.760236 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5103d5-6095-4074-8ce9-0d2c7be940c4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:50.764849 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:50.765493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:50.765830 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:50.766103 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:50.766803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:50.768735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:50.850252 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1370/2778] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:23:50 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:23:50.854193 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:50.864935 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca9269-bead-486c-ad1d-f18527dace94 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:50.924474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:50.925182 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:50.925466 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:50.925794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:50.926513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:50.928539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:51.024079 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1409/2779] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:23:50 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:23:51.028255 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:51.039346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61a76e-36cd-462e-bc8d-44c1f9a1de9c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:51.044160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:51.044823 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:51.045111 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:51.045391 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:51.046118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:51.048399 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:51.065263 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1371/2780] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:51.066499 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:51.076476 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f931d7f-b027-4661-b494-3b84be76ce8b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:51.081185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd3272-2ff0-4aed-b0d8-35c68f932018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:51.081841 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd3272-2ff0-4aed-b0d8-35c68f932018 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:51.082127 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd3272-2ff0-4aed-b0d8-35c68f932018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:51.082405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd3272-2ff0-4aed-b0d8-35c68f932018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:51.083123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd3272-2ff0-4aed-b0d8-35c68f932018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:51.084770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd3272-2ff0-4aed-b0d8-35c68f932018 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:51.097002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1410/2781] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:23:51.101132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:51.111944 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af48ed-3f09-4314-8936-f8968a011163 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:51.116438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:51.117309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:51.117587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:51.117856 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] PATH_INFO: `/v3/projects/6ca58988a48346838ec86ccc4e714484/users/7f1dfd51155b40a7b9402351a01ead69/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:51.131911 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca58988a48346838ec86ccc4e714484, user_id=7f1dfd51155b40a7b9402351a01ead69, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:51.134589 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:51.226180 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1372/2782] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:23:51 2022] PUT /identity/v3/projects/6ca58988a48346838ec86ccc4e714484/users/7f1dfd51155b40a7b9402351a01ead69/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:23:51.237757 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400109-8dbd-47c8-b933-339bb84d9305 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:51.238045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400109-8dbd-47c8-b933-339bb84d9305 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:51.238318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400109-8dbd-47c8-b933-339bb84d9305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:51.265149 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-63400109-8dbd-47c8-b933-339bb84d9305 None None] MFA Rules not processed for user `7f1dfd51155b40a7b9402351a01ead69`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:23:51.266416 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-63400109-8dbd-47c8-b933-339bb84d9305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:51.330305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1411/2783] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:23:51.340829 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:23:51.342493 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:23:51.401821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810944-9f86-401b-b44c-8bf3ff3bdd12 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:23:51.405455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5741c8-2e44-4b6f-8699-2a6ef5b9e68d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:23:51.406182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5741c8-2e44-4b6f-8699-2a6ef5b9e68d service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:23:51.407105 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5741c8-2e44-4b6f-8699-2a6ef5b9e68d service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:23:51.407440 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5741c8-2e44-4b6f-8699-2a6ef5b9e68d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:23:51.421858 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5741c8-2e44-4b6f-8699-2a6ef5b9e68d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:23:51.426339 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5741c8-2e44-4b6f-8699-2a6ef5b9e68d service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:23:51.438986 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1373/2784] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:23:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:24:02.516604 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63400109-8dbd-47c8-b933-339bb84d9305 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:02.577369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63400109-8dbd-47c8-b933-339bb84d9305 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:02.581826 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:02.582725 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:02.583006 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:02.583276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] PATH_INFO: `/v3/users/7f1dfd51155b40a7b9402351a01ead69` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:02.584484 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1dfd51155b40a7b9402351a01ead69)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:02.586396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:02.749194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1dfd51155b40a7b9402351a01ead69'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:02.810771 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1dfd51155b40a7b9402351a01ead69'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:02.817889 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1dfd51155b40a7b9402351a01ead69'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:02.818590 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1dfd51155b40a7b9402351a01ead69'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:02.830932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1dfd51155b40a7b9402351a01ead69'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:02.836817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1412/2785] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:24:02 2022] DELETE /identity/v3/users/7f1dfd51155b40a7b9402351a01ead69 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:24:02.841533 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:02.842070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:02.842070 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:02.859894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:02.861073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:02.917626 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1374/2786] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:02.927374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:02.928813 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:02.983990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcc1f3-7271-4b3e-b639-9039eb773ec0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:02.988010 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fca46-718f-417a-a5bf-1ea1089f0b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:02.988664 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fca46-718f-417a-a5bf-1ea1089f0b05 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:02.988924 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fca46-718f-417a-a5bf-1ea1089f0b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:02.989198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fca46-718f-417a-a5bf-1ea1089f0b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:02.998091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fca46-718f-417a-a5bf-1ea1089f0b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:02.999645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fca46-718f-417a-a5bf-1ea1089f0b05 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:03.009650 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1413/2787] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:24:03.466917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:03.469941 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:03.480613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeccf6-c26b-4d5f-8f2b-9d5f4a0967b5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:03.486061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:03.487313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:03.487313 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:03.487547 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] PATH_INFO: `/v3/projects/6ca58988a48346838ec86ccc4e714484` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:03.488698 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca58988a48346838ec86ccc4e714484)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:03.491004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:03.601240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca58988a48346838ec86ccc4e714484', 'request_id': 'req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:03.667804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca58988a48346838ec86ccc4e714484', 'request_id': 'req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:03.673573 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca58988a48346838ec86ccc4e714484', 'request_id': 'req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:03.684359 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] The token cache is being invalidate because project 6ca58988a48346838ec86ccc4e714484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:24:03.684646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:03.685539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34d1878-7b2a-48d8-a5d6-357ad76c64d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:03.687125 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1375/2788] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:24:03 2022] DELETE /identity/v3/projects/6ca58988a48346838ec86ccc4e714484 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:24:03.692976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277e4c-123b-4226-8410-7f2f5a789e10 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:03.693616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277e4c-123b-4226-8410-7f2f5a789e10 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:03.693616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277e4c-123b-4226-8410-7f2f5a789e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:03.711835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-31277e4c-123b-4226-8410-7f2f5a789e10 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:03.713636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31277e4c-123b-4226-8410-7f2f5a789e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:03.772457 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1414/2789] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:03.780279 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4356d-3504-4789-9af0-9413786cfc12 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:03.780567 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4356d-3504-4789-9af0-9413786cfc12 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:03.780845 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4356d-3504-4789-9af0-9413786cfc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:03.798846 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4ca4356d-3504-4789-9af0-9413786cfc12 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:03.800088 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4ca4356d-3504-4789-9af0-9413786cfc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:03.807914 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1376/2790] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:03.811990 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277e4c-123b-4226-8410-7f2f5a789e10 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:03.822224 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277e4c-123b-4226-8410-7f2f5a789e10 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:03.826742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:03.827389 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:03.827702 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:03.827983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:03.829054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:24:03.829362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:03.830954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:03.842116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1415/2791] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:24:03.846860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4356d-3504-4789-9af0-9413786cfc12 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:03.856135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4356d-3504-4789-9af0-9413786cfc12 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:03.860680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:03.861321 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:03.861600 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:03.861870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:03.862556 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:03.864482 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:03.943919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1377/2792] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:24:03 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:24:03.948179 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:03.958948 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411501ec-9ef9-48d9-8ea2-d40a0e8e6623 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:04.012147 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:04.012809 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:04.013087 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:04.013357 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:04.014047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:04.016595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:04.070405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1416/2793] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:24:03 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:24:04.074525 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:04.085303 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1d98-9616-4a84-9438-45649c03893d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:04.090084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:04.090744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:04.091027 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:04.091304 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:04.092102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:04.093749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:04.104551 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1378/2794] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:24:04.110085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:04.118464 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8c6b0-2e50-4a7b-801d-f56b78a54c33 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:04.123081 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd60902-886c-4fcd-b8d5-010ef3a60a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:04.123758 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd60902-886c-4fcd-b8d5-010ef3a60a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:04.124045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd60902-886c-4fcd-b8d5-010ef3a60a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:04.124322 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd60902-886c-4fcd-b8d5-010ef3a60a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:04.125588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd60902-886c-4fcd-b8d5-010ef3a60a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:04.127373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd60902-886c-4fcd-b8d5-010ef3a60a0a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:04.136950 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1417/2795] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:24:04.141585 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:04.151820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f406cec-3e60-4da9-b425-c8b134539d0b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:04.156657 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:04.157580 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:04.157866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:04.158147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] PATH_INFO: `/v3/projects/ae0f47501cc04e37bec6bfc3cd696ea9/users/6cda1596ad5d4032827f9250a17f1235/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:04.172741 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0f47501cc04e37bec6bfc3cd696ea9, user_id=6cda1596ad5d4032827f9250a17f1235, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:04.175532 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:04.241733 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1379/2796] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:24:04 2022] PUT /identity/v3/projects/ae0f47501cc04e37bec6bfc3cd696ea9/users/6cda1596ad5d4032827f9250a17f1235/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:24:04.246694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597b014-33a2-423b-b3bf-b253e7837786 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:04.247099 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597b014-33a2-423b-b3bf-b253e7837786 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:04.247411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597b014-33a2-423b-b3bf-b253e7837786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:04.266405 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1597b014-33a2-423b-b3bf-b253e7837786 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:04.267743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1597b014-33a2-423b-b3bf-b253e7837786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:04.325988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1418/2797] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:04.335486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:04.337158 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:04.391330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a16d4a-9b43-448d-af83-5bf50e373acc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:04.394885 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:04.395517 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:04.395824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:04.396096 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:04.404999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:04.406560 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:04.418779 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1380/2798] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:24:09.294600 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394abb9f-a2b5-4840-a5e3-366127935ad9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:09.295046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394abb9f-a2b5-4840-a5e3-366127935ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:09.295142 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394abb9f-a2b5-4840-a5e3-366127935ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:09.324793 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-394abb9f-a2b5-4840-a5e3-366127935ad9 None None] MFA Rules not processed for user `6cda1596ad5d4032827f9250a17f1235`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:09.326056 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-394abb9f-a2b5-4840-a5e3-366127935ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:09.411576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1419/2799] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:09.412841 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:09.423329 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a94d2a-20e0-4b0e-af3e-cd6bf6282cad service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:09.429072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:09.429072 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:09.429380 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:09.429527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:09.430230 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:09.432169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:09.541328 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1381/2800] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:24:09 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:24:09.545144 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394abb9f-a2b5-4840-a5e3-366127935ad9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:09.560205 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394abb9f-a2b5-4840-a5e3-366127935ad9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:09.619572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:09.620320 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:09.620606 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:09.620885 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:09.621598 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:09.623618 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:09.717111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1420/2801] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:24:09 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:24:09.719145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:09.730259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bd4b-6367-4a73-9a14-8a9c0a654893 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:09.736215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:09.736902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:09.737182 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:09.737454 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:09.738176 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:09.739939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:09.752616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1382/2802] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:24:09.756835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:09.771233 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd24f1-7e20-43de-ad3a-9df00da9dc19 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:09.782249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:09.799940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:09.800240 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:09.800575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] PATH_INFO: `/v3/projects/33b8cf1c5afb40fa83960b70b30de818/users/679f76cc5c67448e91b7e7abcff95a1c/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:09.817753 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b8cf1c5afb40fa83960b70b30de818, user_id=679f76cc5c67448e91b7e7abcff95a1c, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:09.820465 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:09.920451 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1421/2803] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:24:09 2022] PUT /identity/v3/projects/33b8cf1c5afb40fa83960b70b30de818/users/679f76cc5c67448e91b7e7abcff95a1c/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:24:09.925476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:09.944706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9e02e-7084-4170-b944-30c9312324f9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:10.010827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29cad0-f9f2-40f7-b342-67a7c6170fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:10.011919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29cad0-f9f2-40f7-b342-67a7c6170fee admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:10.012521 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29cad0-f9f2-40f7-b342-67a7c6170fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:10.012954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29cad0-f9f2-40f7-b342-67a7c6170fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:10.013839 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29cad0-f9f2-40f7-b342-67a7c6170fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:10.016133 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29cad0-f9f2-40f7-b342-67a7c6170fee admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:10.027302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1383/2804] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:24:10.033004 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:10.046175 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444e352-112c-4587-91c6-469315d8b166 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:10.055117 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:10.056053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:10.056335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:10.056610 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] PATH_INFO: `/v3/projects/33b8cf1c5afb40fa83960b70b30de818/users/679f76cc5c67448e91b7e7abcff95a1c/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:10.058619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b8cf1c5afb40fa83960b70b30de818, user_id=679f76cc5c67448e91b7e7abcff95a1c, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:10.061316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:10.076765 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1422/2805] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:24:10 2022] PUT /identity/v3/projects/33b8cf1c5afb40fa83960b70b30de818/users/679f76cc5c67448e91b7e7abcff95a1c/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:24:15.724966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9bf2-1921-4d8c-8985-98d0c703494e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:15.725785 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9bf2-1921-4d8c-8985-98d0c703494e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:15.726281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9bf2-1921-4d8c-8985-98d0c703494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:15.753511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-aafc9bf2-1921-4d8c-8985-98d0c703494e None None] MFA Rules not processed for user `679f76cc5c67448e91b7e7abcff95a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:15.754795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-aafc9bf2-1921-4d8c-8985-98d0c703494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:15.824418 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1384/2806] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:15.836901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:15.838620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:15.901563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af8a88-98fe-40c5-852b-ce45ee145fcb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:15.905238 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60d4a75e5b8464a8c7935e7433cf170', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:15.905891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:15.906759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:15.907030 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:15.966315 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:15.967829 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:15.977575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1423/2807] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:24:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:24:16.204157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:16.204730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:16.205086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:16.224721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:16.225935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:16.300056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1385/2808] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:24:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:16.313213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:16.315092 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:16.393430 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ab51e-573c-43a2-a8bc-44fb4d535905 service glance] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:16.399243 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50868998-dd2f-4c6d-8cfe-1fc253108857 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:16.400352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50868998-dd2f-4c6d-8cfe-1fc253108857 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:16.400748 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50868998-dd2f-4c6d-8cfe-1fc253108857 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:16.401109 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50868998-dd2f-4c6d-8cfe-1fc253108857 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:16.410979 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50868998-dd2f-4c6d-8cfe-1fc253108857 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:16.412774 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50868998-dd2f-4c6d-8cfe-1fc253108857 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:16.422409 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1424/2809] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:24:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:24:19.544416 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:19.545948 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:19.687073 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ecce3-498d-4b5d-a6ce-d9a794b5602b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:19.693956 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:19.694623 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:19.694902 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:19.695172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:19.697503 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:19.712438 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:19.713697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:19.731820 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1386/2810] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:24:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:24:26.397278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d9511-c9a3-4753-bd54-a89e66f517dc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:26.410474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d9511-c9a3-4753-bd54-a89e66f517dc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:26.410474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d9511-c9a3-4753-bd54-a89e66f517dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:26.414693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6a6d9511-c9a3-4753-bd54-a89e66f517dc None None] MFA Rules not processed for user `6cda1596ad5d4032827f9250a17f1235`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:26.415991 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6a6d9511-c9a3-4753-bd54-a89e66f517dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:26.423333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1425/2811] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:26.994998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:26.996410 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:27.064564 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e85fc3-5c2a-4c24-8328-ee3092204b34 service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:27.069155 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:27.070034 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:27.070309 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:27.070572 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] PATH_INFO: `/v3/users/ae7b0bf112754991b243bc4cc8598231` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:27.071793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7b0bf112754991b243bc4cc8598231)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:27.073706 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:27.299026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7b0bf112754991b243bc4cc8598231'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:27.361569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7b0bf112754991b243bc4cc8598231'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:27.362131 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7b0bf112754991b243bc4cc8598231'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:27.374175 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7b0bf112754991b243bc4cc8598231'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:27.381302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7b0bf112754991b243bc4cc8598231'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:27.387285 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1387/2812] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:24:26 2022] DELETE /identity/v3/users/ae7b0bf112754991b243bc4cc8598231 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:24:27.392444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:27.392740 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:27.393024 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:27.411223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:27.412564 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:27.471007 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1426/2813] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:27.480680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:27.482114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:27.540150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c91b-df68-437d-bc37-955fc51a8e36 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:27.543688 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d077c54-696f-41ee-922f-2724c675aec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:27.544335 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d077c54-696f-41ee-922f-2724c675aec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:27.544619 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d077c54-696f-41ee-922f-2724c675aec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:27.544890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d077c54-696f-41ee-922f-2724c675aec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:27.553870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d077c54-696f-41ee-922f-2724c675aec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:27.555306 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d077c54-696f-41ee-922f-2724c675aec1 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:27.564604 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1388/2814] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:24:27.969153 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:27.971655 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:27.984377 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfd446-cc72-45ae-be8f-d625e4bc5dc5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:27.988967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:27.990047 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:27.990396 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:27.990679 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] PATH_INFO: `/v3/projects/9c06bd5ee6e64ed08b78f9f22727e7d5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:27.991976 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c06bd5ee6e64ed08b78f9f22727e7d5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:27.993931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:28.130433 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c06bd5ee6e64ed08b78f9f22727e7d5', 'request_id': 'req-52dd33a4-83d8-4fdc-8f22-1526ddedf369'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:28.135862 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c06bd5ee6e64ed08b78f9f22727e7d5', 'request_id': 'req-52dd33a4-83d8-4fdc-8f22-1526ddedf369'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:28.142219 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c06bd5ee6e64ed08b78f9f22727e7d5', 'request_id': 'req-52dd33a4-83d8-4fdc-8f22-1526ddedf369'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:28.229709 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] The token cache is being invalidate because project 9c06bd5ee6e64ed08b78f9f22727e7d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:24:28.229910 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:28.230512 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:24:28.232104 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1427/2815] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:24:27 2022] DELETE /identity/v3/projects/9c06bd5ee6e64ed08b78f9f22727e7d5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:24:32.863827 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a60ec-5027-427b-b2f8-e02daa83e0d2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:32.864344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a60ec-5027-427b-b2f8-e02daa83e0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:32.864690 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a60ec-5027-427b-b2f8-e02daa83e0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:32.883854 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-665a60ec-5027-427b-b2f8-e02daa83e0d2 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:32.885064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-665a60ec-5027-427b-b2f8-e02daa83e0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:32.940753 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1389/2816] 10.222.0.71 () {58 vars in 1009 bytes} [Mon May 16 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:24:32.963249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:24:32.964790 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:33.035310 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd33a4-83d8-4fdc-8f22-1526ddedf369 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:24:33.040863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:24:33.041520 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:33.041803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:33.042080 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:33.050978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:24:33.052767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:24:33.060897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1428/2817] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:24:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:24:51.983994 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:24:51.984524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:24:51.984524 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:24:52.004320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] MFA Rules not processed for user `d37b3b7824b24b328138061ae7af38f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:24:52.005713 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:24:52.060157 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1390/2818] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:25:03.725504 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:25:03.726940 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:25:03.783198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dba8d-3df7-4647-98ec-28fffa225ddb service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:25:03.786674 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:25:03.787317 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:25:03.787599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:25:03.787898 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:25:03.789693 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:25:03.849707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:25:03.851176 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:25:03.860482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1429/2819] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:25:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:25:27.462527 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:25:27.463932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:25:27.519324 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3fa12-9b79-4741-a35e-745f18a1f853 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:25:27.523142 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:25:27.523814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:25:27.524102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:25:27.524382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:25:27.526186 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:25:27.593980 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:25:27.595407 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:25:27.606664 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1391/2820] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:25:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:25:30.520819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:25:30.529551 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddf44a-cb69-4c67-9962-425006c3b555 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:25:30.533192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:25:30.533830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:25:30.534120 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:25:30.534573 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:25:30.536270 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:25:30.549478 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:25:30.551095 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:25:30.562017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1430/2821] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:25:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:26:00.042870 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:00.053200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:00.107241 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66559725-1a7f-4dcc-9c87-009ee5b7e3e9 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:00.111888 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:00.112757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:00.113038 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:00.113314 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] PATH_INFO: `/v3/users/d37b3b7824b24b328138061ae7af38f8` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:00.114591 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37b3b7824b24b328138061ae7af38f8)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:00.116569 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:00.249882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37b3b7824b24b328138061ae7af38f8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.299697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37b3b7824b24b328138061ae7af38f8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.300240 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37b3b7824b24b328138061ae7af38f8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.312411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37b3b7824b24b328138061ae7af38f8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.318894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7c98b73e-0f04-419e-aa5a-98cced3ebdf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37b3b7824b24b328138061ae7af38f8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.325788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1392/2822] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:26:00 2022] DELETE /identity/v3/users/d37b3b7824b24b328138061ae7af38f8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:00.564362 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:00.575868 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138d323-e340-4768-b4fa-1cfc8dc5d23d service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:00.637616 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:00.638474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:00.638776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:00.639049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] PATH_INFO: `/v3/projects/5e343d0e69de4a2c956a7ebc9056b8d5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:00.640235 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e343d0e69de4a2c956a7ebc9056b8d5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:00.642168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:00.762768 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e343d0e69de4a2c956a7ebc9056b8d5', 'request_id': 'req-bc883674-fb35-43ad-92e5-199c9a3aed6a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.768217 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e343d0e69de4a2c956a7ebc9056b8d5', 'request_id': 'req-bc883674-fb35-43ad-92e5-199c9a3aed6a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.774536 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e343d0e69de4a2c956a7ebc9056b8d5', 'request_id': 'req-bc883674-fb35-43ad-92e5-199c9a3aed6a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.840644 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] The token cache is being invalidate because project 5e343d0e69de4a2c956a7ebc9056b8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:26:00.840927 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.841795 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc883674-fb35-43ad-92e5-199c9a3aed6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:00.843517 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1431/2823] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:26:00 2022] DELETE /identity/v3/projects/5e343d0e69de4a2c956a7ebc9056b8d5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:00.848424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482e33-5a77-4e99-9820-a0e859fb3bc2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:00.848746 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482e33-5a77-4e99-9820-a0e859fb3bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:00.849050 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482e33-5a77-4e99-9820-a0e859fb3bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:00.867907 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-54482e33-5a77-4e99-9820-a0e859fb3bc2 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:26:00.869697 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-54482e33-5a77-4e99-9820-a0e859fb3bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:00.926592 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1393/2824] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:26:00.934507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25466b13-b7f9-4a60-8fc6-8eec303fcd79 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:00.934802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25466b13-b7f9-4a60-8fc6-8eec303fcd79 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:00.935085 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25466b13-b7f9-4a60-8fc6-8eec303fcd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:00.955088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-25466b13-b7f9-4a60-8fc6-8eec303fcd79 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:26:00.956319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-25466b13-b7f9-4a60-8fc6-8eec303fcd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:00.965620 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1432/2825] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:26:00.969816 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54482e33-5a77-4e99-9820-a0e859fb3bc2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:00.979763 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54482e33-5a77-4e99-9820-a0e859fb3bc2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:00.984472 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:00.985127 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:00.985408 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:00.985725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:00.986430 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:26:00.986729 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:00.988361 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:00.999516 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1394/2826] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:26:01.003583 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25466b13-b7f9-4a60-8fc6-8eec303fcd79 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:01.013116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25466b13-b7f9-4a60-8fc6-8eec303fcd79 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:01.018018 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:01.018696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:01.018975 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:01.019246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:01.020234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:01.022105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:01.147050 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1433/2827] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:26:01 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:26:01.152327 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:01.162247 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade987e-b928-4621-993e-2cc04d6dd0f9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:01.215919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:01.216575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:01.216860 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:01.217134 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:01.217811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:01.219811 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:01.301991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1395/2828] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:26:01 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:26:01.306779 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:01.317168 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65dbff-ebce-4dbe-9568-051053e8039e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:01.322411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:01.323059 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:01.323338 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:01.323611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:01.324412 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:01.326002 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:01.336411 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1434/2829] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:26:01.341692 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:01.350708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894da87-1c21-4d37-9548-284d154fd185 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:01.356107 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9184569-0019-49b8-807d-c5787e31cd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:01.356757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184569-0019-49b8-807d-c5787e31cd2e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:01.357037 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184569-0019-49b8-807d-c5787e31cd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:01.357311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184569-0019-49b8-807d-c5787e31cd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:01.358054 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9184569-0019-49b8-807d-c5787e31cd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:01.359646 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9184569-0019-49b8-807d-c5787e31cd2e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:01.369320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1396/2830] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:26:01.373415 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:01.383232 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38243d-e15c-4c31-af40-0fe192104840 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:01.388027 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:01.388894 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:01.389171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:01.389441 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] PATH_INFO: `/v3/projects/5cef808bf8324f5b852fd0ea7f368f6d/users/3715d415c2e24297a30a4c948c2677ef/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:01.405025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cef808bf8324f5b852fd0ea7f368f6d, user_id=3715d415c2e24297a30a4c948c2677ef, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:01.407691 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:01.496611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1435/2831] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:26:01 2022] PUT /identity/v3/projects/5cef808bf8324f5b852fd0ea7f368f6d/users/3715d415c2e24297a30a4c948c2677ef/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:01.501588 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e2ed-d830-4f3a-af2c-81458191fe9f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:01.507575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e2ed-d830-4f3a-af2c-81458191fe9f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:01.507575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e2ed-d830-4f3a-af2c-81458191fe9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:01.519488 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3506e2ed-d830-4f3a-af2c-81458191fe9f None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:26:01.520966 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3506e2ed-d830-4f3a-af2c-81458191fe9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:01.578518 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1397/2832] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:26:01.588576 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:01.590276 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:01.644849 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0f4de-576c-47cd-b20e-988bbfcc0bc0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:01.648724 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:01.649955 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:01.650228 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:01.650500 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:01.659364 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:01.660767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:01.670123 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1436/2833] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:26:06.631448 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247e2c1-da11-4282-ac25-f1f7a5bf0a57 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:06.631983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247e2c1-da11-4282-ac25-f1f7a5bf0a57 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:06.632082 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247e2c1-da11-4282-ac25-f1f7a5bf0a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:06.662275 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-7247e2c1-da11-4282-ac25-f1f7a5bf0a57 None None] MFA Rules not processed for user `3715d415c2e24297a30a4c948c2677ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:26:06.663504 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7247e2c1-da11-4282-ac25-f1f7a5bf0a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:06.734709 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1398/2834] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:26:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:26:06.739165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:06.753170 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fc13-a6a2-4be7-841b-35f8e7de8a0f service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:06.757891 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:06.759360 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:06.759817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:06.760278 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:06.760988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:06.763537 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:06.848728 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1437/2835] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:26:06 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:26:06.849937 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247e2c1-da11-4282-ac25-f1f7a5bf0a57 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:06.861726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247e2c1-da11-4282-ac25-f1f7a5bf0a57 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:06.921115 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:06.921927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:06.922222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:06.922492 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:06.923200 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:06.925222 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:07.014059 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1399/2836] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:26:06 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 15:26:07.017895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:07.028743 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec2122-6f81-498f-bdf1-d909fa37b84d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:07.033375 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:07.034032 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:07.034316 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:07.034595 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:07.035318 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:07.036987 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:07.050554 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1438/2837] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:26:07.055281 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:07.066710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9ceac-07fa-4bd2-928b-aaab4e54ee81 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:07.071768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:07.072725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:07.073008 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:07.073284 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] PATH_INFO: `/v3/projects/504104ad8b9048a2bb1e1140fceab85c/users/fb30663053ac4b2095255b42bba259e3/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:07.089084 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504104ad8b9048a2bb1e1140fceab85c, user_id=fb30663053ac4b2095255b42bba259e3, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:07.091912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:07.220891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1400/2838] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:26:07 2022] PUT /identity/v3/projects/504104ad8b9048a2bb1e1140fceab85c/users/fb30663053ac4b2095255b42bba259e3/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:07.224723 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:07.236044 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10532cd9-bb05-42b3-8765-24e06d6df649 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:07.293848 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b17f56-cfa1-4b24-87fe-d1d64b63a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:07.294527 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b17f56-cfa1-4b24-87fe-d1d64b63a146 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:07.294817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b17f56-cfa1-4b24-87fe-d1d64b63a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:07.295096 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b17f56-cfa1-4b24-87fe-d1d64b63a146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:07.295879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b17f56-cfa1-4b24-87fe-d1d64b63a146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:07.297494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b17f56-cfa1-4b24-87fe-d1d64b63a146 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:07.307936 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1439/2839] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 15:26:07.312056 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:07.323227 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d96d1f-6f83-44a9-8625-bf9d5d5bc9f5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:07.328015 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:07.328909 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:07.329190 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:07.329463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] PATH_INFO: `/v3/projects/504104ad8b9048a2bb1e1140fceab85c/users/fb30663053ac4b2095255b42bba259e3/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:07.332468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504104ad8b9048a2bb1e1140fceab85c, user_id=fb30663053ac4b2095255b42bba259e3, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:07.335204 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:07.346960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1401/2840] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:26:07 2022] PUT /identity/v3/projects/504104ad8b9048a2bb1e1140fceab85c/users/fb30663053ac4b2095255b42bba259e3/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:12.679105 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b24ac-f8e7-472b-ae56-4d13de0b94d8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:12.679683 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b24ac-f8e7-472b-ae56-4d13de0b94d8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:12.680034 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b24ac-f8e7-472b-ae56-4d13de0b94d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:12.706453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ea9b24ac-f8e7-472b-ae56-4d13de0b94d8 None None] MFA Rules not processed for user `fb30663053ac4b2095255b42bba259e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:26:12.707690 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ea9b24ac-f8e7-472b-ae56-4d13de0b94d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:12.779908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1440/2841] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:26:12.793852 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:12.796064 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:12.878010 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade493-6338-40f1-9856-e2f88a8f8e36 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:12.884251 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:12.884919 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:12.885243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:12.885476 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:12.961355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:12.963051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:12.971815 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1402/2842] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:26:16.446432 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2fd44-56ff-4db6-af25-459de977bcd7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:16.447319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2fd44-56ff-4db6-af25-459de977bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:16.447319 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2fd44-56ff-4db6-af25-459de977bcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:16.465884 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-23f2fd44-56ff-4db6-af25-459de977bcd7 None None] MFA Rules not processed for user `3715d415c2e24297a30a4c948c2677ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:26:16.467453 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23f2fd44-56ff-4db6-af25-459de977bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:16.475025 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1441/2843] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:26:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:26:19.170539 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:19.172311 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:19.246162 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be71f6b-8164-4025-a2e6-602b6c50ef90 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:19.249917 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78ddf9-a97d-427d-b802-7f264dbb6f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:19.251215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78ddf9-a97d-427d-b802-7f264dbb6f7c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:19.251717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78ddf9-a97d-427d-b802-7f264dbb6f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:19.252032 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78ddf9-a97d-427d-b802-7f264dbb6f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:19.255147 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0c78ddf9-a97d-427d-b802-7f264dbb6f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:19.269523 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78ddf9-a97d-427d-b802-7f264dbb6f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:19.271426 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78ddf9-a97d-427d-b802-7f264dbb6f7c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:19.280662 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1403/2844] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:26:20.484791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2fd44-56ff-4db6-af25-459de977bcd7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:20.500646 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2fd44-56ff-4db6-af25-459de977bcd7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:20.506705 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:20.507379 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:20.508174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:20.508174 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:20.511214 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:20.524671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:20.526599 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:20.538475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1442/2845] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:26:30.466451 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:30.466890 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:30.466983 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:30.484727 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] MFA Rules not processed for user `4dd72a14691c48dd95845b7b46eb8c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:26:30.485960 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:30.537536 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1404/2846] 10.222.0.71 () {58 vars in 967 bytes} [Mon May 16 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:26:30.578603 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:30.580892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:30.647547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a204ce-0c62-4baf-9852-0d4672ece7b4 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:30.653012 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918d67038f9c4d0fa4d2456409245774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:30.653652 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:30.653932 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:30.654207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:30.662082 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:30.663817 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:30.672246 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1443/2847] 10.222.0.71 () {60 vars in 1384 bytes} [Mon May 16 15:26:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 15:26:34.176999 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:34.183141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:34.275612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5f985-1983-4b20-a3b7-4d6f097ca8f5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:34.287957 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:34.288641 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:34.288922 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:34.289271 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:34.292392 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:34.305453 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:34.307132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:34.318118 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1405/2848] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:26:38.587731 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:38.589486 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:38.609353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9617b-5e20-417a-b7f2-8a11ae72f3b8 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:38.615249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:38.616017 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:38.616401 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:38.616747 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:38.619495 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:38.632897 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:38.634767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:38.646734 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1444/2849] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:26:38.663765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:38.666461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:38.768954 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b79afc-78fd-4390-8630-a7152b3a0bd3 service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:38.773768 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:38.775272 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:38.775550 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:38.775859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] PATH_INFO: `/v3/users/6cda1596ad5d4032827f9250a17f1235` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:38.777057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cda1596ad5d4032827f9250a17f1235)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:38.779003 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:38.983460 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cda1596ad5d4032827f9250a17f1235'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.064735 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cda1596ad5d4032827f9250a17f1235'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.065395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cda1596ad5d4032827f9250a17f1235'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.079074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cda1596ad5d4032827f9250a17f1235'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.085794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cda1596ad5d4032827f9250a17f1235'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.091991 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1406/2850] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:26:38 2022] DELETE /identity/v3/users/6cda1596ad5d4032827f9250a17f1235 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:39.329426 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:39.342804 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5668d-0a92-44d2-a608-7078c23a7b8b service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:39.405901 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:39.407053 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:39.407334 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:39.407694 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] PATH_INFO: `/v3/projects/ae0f47501cc04e37bec6bfc3cd696ea9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:39.408928 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0f47501cc04e37bec6bfc3cd696ea9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:39.411210 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:39.542440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0f47501cc04e37bec6bfc3cd696ea9', 'request_id': 'req-56be6583-4326-41ff-aebb-297946cace59'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.547859 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0f47501cc04e37bec6bfc3cd696ea9', 'request_id': 'req-56be6583-4326-41ff-aebb-297946cace59'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.554843 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0f47501cc04e37bec6bfc3cd696ea9', 'request_id': 'req-56be6583-4326-41ff-aebb-297946cace59'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.644049 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] The token cache is being invalidate because project ae0f47501cc04e37bec6bfc3cd696ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:26:39.644967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.645503 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:39.647130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1445/2851] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:26:39 2022] DELETE /identity/v3/projects/ae0f47501cc04e37bec6bfc3cd696ea9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:39.651502 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:39.652873 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:39.666138 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e69aeb-f899-46a1-9820-d46294eb8418 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:39.725520 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:39.726398 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:39.726821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:39.727102 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] PATH_INFO: `/v3/users/679f76cc5c67448e91b7e7abcff95a1c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:39.729075 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=679f76cc5c67448e91b7e7abcff95a1c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:39.731927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:40.098509 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '679f76cc5c67448e91b7e7abcff95a1c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.162291 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '679f76cc5c67448e91b7e7abcff95a1c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.162942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '679f76cc5c67448e91b7e7abcff95a1c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.175203 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '679f76cc5c67448e91b7e7abcff95a1c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.181749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '679f76cc5c67448e91b7e7abcff95a1c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.187810 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1407/2852] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:26:39 2022] DELETE /identity/v3/users/679f76cc5c67448e91b7e7abcff95a1c => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:40.414601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:40.416900 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:40.428755 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be6583-4326-41ff-aebb-297946cace59 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:40.487833 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:40.488738 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:40.489021 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:40.489296 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] PATH_INFO: `/v3/projects/33b8cf1c5afb40fa83960b70b30de818` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:40.490467 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b8cf1c5afb40fa83960b70b30de818)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:40.492408 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:40.619721 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b8cf1c5afb40fa83960b70b30de818', 'request_id': 'req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.624890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b8cf1c5afb40fa83960b70b30de818', 'request_id': 'req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.630761 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b8cf1c5afb40fa83960b70b30de818', 'request_id': 'req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.735505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] The token cache is being invalidate because project 33b8cf1c5afb40fa83960b70b30de818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:26:40.736169 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.736447 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:26:40.738020 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1446/2853] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:26:40 2022] DELETE /identity/v3/projects/33b8cf1c5afb40fa83960b70b30de818 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:26:48.882235 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:48.884039 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:49.008341 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b8b8f-23d0-45f5-a465-8e7e5db173eb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:49.069724 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:49.070382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:49.070670 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:49.070973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:49.073374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:49.085571 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:49.087177 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:49.098558 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1408/2854] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:26:52.225636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:26:52.227106 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:52.306988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6d051-3d0e-4800-a072-59d5a2c9dcf6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:26:52.310611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:26:52.311252 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:26:52.311539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:26:52.311844 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:26:52.313615 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:26:52.373818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:26:52.375251 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:26:52.384686 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1447/2855] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:26:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:29:03.387302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:29:03.397046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649c908-2ed1-42f4-9ef0-e975c5247f7d service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:29:03.400610 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:29:03.401243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:29:03.401528 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:29:03.401804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:29:03.404290 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:29:03.413711 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:29:03.415132 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:29:03.423659 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1409/2856] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:29:03.756116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:29:03.757780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:29:03.769830 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d10c-025d-4982-affa-a348b63b070e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:29:03.773440 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:29:03.774116 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:29:03.774402 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:29:03.774682 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:29:03.776507 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:29:03.787474 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:29:03.789054 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:29:03.797340 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1448/2857] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:29:03.868160 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:29:03.876945 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313bc79-cbc6-4351-8af5-2cfc14215694 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:29:03.880562 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:29:03.881199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:29:03.881481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:29:03.881754 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:29:03.883497 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:29:03.916254 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:29:03.917671 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:29:03.937258 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1410/2858] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:29:20.379131 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:29:20.380575 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:29:20.466972 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b761fc0-c224-4466-aa93-f846c50393f4 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:29:20.471943 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:29:20.472582 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:29:20.472863 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:29:20.473137 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:29:20.474892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:29:20.486471 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:29:20.488062 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:29:20.498329 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1449/2859] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:29:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:30:37.164942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:37.175969 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96006d3d-d556-477f-9f1d-762341c151eb service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:37.179893 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:37.181344 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:37.181631 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:37.181906 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:37.185150 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:37.201584 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:37.203346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:37.215537 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1411/2860] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:30:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:30:38.419912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:38.433882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4432-83a3-4aea-9247-daf2ccadd9a1 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:38.438333 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:38.439051 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:38.439455 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:38.439791 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:38.442258 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:38.457352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:38.458802 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:38.475091 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1450/2861] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:30:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:30:49.130308 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:49.133973 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:49.224730 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437c749-6a28-473d-a6dc-e2e2410480a4 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:49.229206 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:49.230061 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:49.230342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:49.230612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] PATH_INFO: `/v3/users/3715d415c2e24297a30a4c948c2677ef` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:49.231800 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3715d415c2e24297a30a4c948c2677ef)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:49.233696 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:49.396436 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3715d415c2e24297a30a4c948c2677ef'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.461192 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3715d415c2e24297a30a4c948c2677ef'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.461901 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3715d415c2e24297a30a4c948c2677ef'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.474089 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3715d415c2e24297a30a4c948c2677ef'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.482838 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3715d415c2e24297a30a4c948c2677ef'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.486435 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1412/2862] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:30:49 2022] DELETE /identity/v3/users/3715d415c2e24297a30a4c948c2677ef => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:30:49.714222 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:49.726374 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f1376-26ad-41f7-9825-15d06a17a03f service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:49.784635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:49.785539 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:49.785819 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:49.786094 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] PATH_INFO: `/v3/projects/5cef808bf8324f5b852fd0ea7f368f6d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:49.787262 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cef808bf8324f5b852fd0ea7f368f6d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:49.789194 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:49.886470 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cef808bf8324f5b852fd0ea7f368f6d', 'request_id': 'req-12ab2bae-adcd-44f4-b96f-40688033cb68'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.891786 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cef808bf8324f5b852fd0ea7f368f6d', 'request_id': 'req-12ab2bae-adcd-44f4-b96f-40688033cb68'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.898241 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cef808bf8324f5b852fd0ea7f368f6d', 'request_id': 'req-12ab2bae-adcd-44f4-b96f-40688033cb68'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.971016 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] The token cache is being invalidate because project 5cef808bf8324f5b852fd0ea7f368f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:30:49.971890 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.972525 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:49.974102 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1451/2863] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:30:49 2022] DELETE /identity/v3/projects/5cef808bf8324f5b852fd0ea7f368f6d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:30:49.978085 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:49.979468 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:49.993025 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db3ee9-5878-4bc9-a4ff-28eea5ca6f9e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:50.046628 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:50.047491 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:50.047786 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:50.048060 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] PATH_INFO: `/v3/users/fb30663053ac4b2095255b42bba259e3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:50.049219 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb30663053ac4b2095255b42bba259e3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:50.051814 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:50.204807 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb30663053ac4b2095255b42bba259e3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.291790 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb30663053ac4b2095255b42bba259e3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.292342 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb30663053ac4b2095255b42bba259e3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.304365 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb30663053ac4b2095255b42bba259e3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.311378 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-42d170c6-41a5-42eb-a816-96e614776761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb30663053ac4b2095255b42bba259e3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.318172 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1413/2864] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:30:49 2022] DELETE /identity/v3/users/fb30663053ac4b2095255b42bba259e3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:30:50.573746 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:50.576547 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:50.589767 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2bae-adcd-44f4-b96f-40688033cb68 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:50.646982 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:50.647886 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:50.648171 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:50.648481 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] PATH_INFO: `/v3/projects/504104ad8b9048a2bb1e1140fceab85c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:50.649667 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504104ad8b9048a2bb1e1140fceab85c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:50.651578 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:50.757545 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504104ad8b9048a2bb1e1140fceab85c', 'request_id': 'req-a242da90-7dd2-4f74-8f56-cf69b978c284'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.762869 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504104ad8b9048a2bb1e1140fceab85c', 'request_id': 'req-a242da90-7dd2-4f74-8f56-cf69b978c284'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.769944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504104ad8b9048a2bb1e1140fceab85c', 'request_id': 'req-a242da90-7dd2-4f74-8f56-cf69b978c284'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.839648 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] The token cache is being invalidate because project 504104ad8b9048a2bb1e1140fceab85c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:30:50.839879 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.840524 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a242da90-7dd2-4f74-8f56-cf69b978c284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:30:50.842086 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1452/2865] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:30:50 2022] DELETE /identity/v3/projects/504104ad8b9048a2bb1e1140fceab85c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:30:50.847199 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8f809-03d5-49f0-836d-93bc849ab259 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:50.847486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8f809-03d5-49f0-836d-93bc849ab259 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:50.847821 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8f809-03d5-49f0-836d-93bc849ab259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:50.905355 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-03a8f809-03d5-49f0-836d-93bc849ab259 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:30:50.906607 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-03a8f809-03d5-49f0-836d-93bc849ab259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:50.962490 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1414/2866] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:30:50.970198 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11810-2e85-47b3-8b75-d91a24192585 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:50.970488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11810-2e85-47b3-8b75-d91a24192585 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:50.970769 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11810-2e85-47b3-8b75-d91a24192585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:50.990459 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f3b11810-2e85-47b3-8b75-d91a24192585 None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:30:50.991764 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f3b11810-2e85-47b3-8b75-d91a24192585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:50.999649 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1453/2867] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:30:51.004315 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8f809-03d5-49f0-836d-93bc849ab259 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:51.013788 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8f809-03d5-49f0-836d-93bc849ab259 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:51.018676 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:51.019169 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.019450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.019760 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.020481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 15:30:51.020783 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:51.022369 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:51.033333 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1415/2868] 10.222.0.71 () {58 vars in 1152 bytes} [Mon May 16 15:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 15:30:51.037549 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b11810-2e85-47b3-8b75-d91a24192585 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:51.046586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b11810-2e85-47b3-8b75-d91a24192585 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:51.051899 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:51.052540 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.052818 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.053090 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.053759 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:51.055653 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:51.185076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1454/2869] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:30:51 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:30:51.189144 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:51.199374 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d6fb-3a23-4350-8f74-a4eb965b37ec admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:51.254209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:51.254895 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.255141 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.255415 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.256145 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:51.258051 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:51.337395 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1416/2870] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:30:51 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:30:51.341522 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:51.352192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a525ff-18ad-463f-bb34-6a4d1abd5f06 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:51.356713 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:51.357356 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.357639 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.357912 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.358626 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:51.360257 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:51.370983 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1455/2871] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:30:51.374803 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:51.384612 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a8f87-d7b4-49c0-8c1f-df14524fd6a6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:51.389212 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e8afe-5ef6-486b-8e91-a1d934839635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:51.389859 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8afe-5ef6-486b-8e91-a1d934839635 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.390137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8afe-5ef6-486b-8e91-a1d934839635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.390409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8afe-5ef6-486b-8e91-a1d934839635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.391122 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e8afe-5ef6-486b-8e91-a1d934839635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:51.392993 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e8afe-5ef6-486b-8e91-a1d934839635 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:51.402708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1417/2872] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:30:51.406645 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:51.417180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7de0a-89c7-42ff-afcc-5dda8be1427e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:51.421775 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:51.422632 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.422916 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.423192 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] PATH_INFO: `/v3/projects/35be16de8a104d569efbb5633c5036af/users/1f3e4b15a6cc4879b24d7a44669a3690/roles/70fe87cb18bc40b69190efb5dd0a933b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.439045 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35be16de8a104d569efbb5633c5036af, user_id=1f3e4b15a6cc4879b24d7a44669a3690, role_id=70fe87cb18bc40b69190efb5dd0a933b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:51.442005 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:51.503036 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1456/2873] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:30:51 2022] PUT /identity/v3/projects/35be16de8a104d569efbb5633c5036af/users/1f3e4b15a6cc4879b24d7a44669a3690/roles/70fe87cb18bc40b69190efb5dd0a933b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:30:51.507998 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba117f3b-b7e8-49c4-a526-a4711758010d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.508294 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba117f3b-b7e8-49c4-a526-a4711758010d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.508575 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba117f3b-b7e8-49c4-a526-a4711758010d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.526228 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ba117f3b-b7e8-49c4-a526-a4711758010d None None] MFA Rules not processed for user `6246a26f46e94e7389a45ee846fec1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:30:51.527461 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ba117f3b-b7e8-49c4-a526-a4711758010d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:51.583367 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1418/2874] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:30:51.593063 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:51.594780 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:51.650588 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a508e-fde9-40b1-b111-6860c8399755 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:51.654223 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:51.654869 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:51.655151 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:51.655425 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:51.665130 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:51.666572 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:51.675954 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1457/2875] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:30:56.848159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86400a43-e768-49bc-a50b-e74ca2d4e98b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:56.848581 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86400a43-e768-49bc-a50b-e74ca2d4e98b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:56.848718 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86400a43-e768-49bc-a50b-e74ca2d4e98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:56.876824 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-86400a43-e768-49bc-a50b-e74ca2d4e98b None None] MFA Rules not processed for user `1f3e4b15a6cc4879b24d7a44669a3690`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:30:56.878714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-86400a43-e768-49bc-a50b-e74ca2d4e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:30:56.945007 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1419/2876] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:30:56.949508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:56.959831 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bdf8b-01ea-480c-92a1-3ad70ae271a1 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:56.964696 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:56.965369 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:56.965671 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:56.965931 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:56.966624 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:56.968670 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:57.089837 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1458/2877] 10.222.0.71 () {60 vars in 1152 bytes} [Mon May 16 15:30:56 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:30:57.090021 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86400a43-e768-49bc-a50b-e74ca2d4e98b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:57.102721 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86400a43-e768-49bc-a50b-e74ca2d4e98b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:57.163864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:57.164546 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:57.164833 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:57.165106 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:57.165795 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:57.167749 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:57.278864 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1420/2878] 10.222.0.71 () {60 vars in 1143 bytes} [Mon May 16 15:30:57 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 15:30:57.282822 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:57.293895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27bbb5-ccc1-4c2d-a46a-001d48ff34ca admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:57.298635 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:57.299875 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:57.300234 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:57.300601 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:57.301348 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:57.302967 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:57.314133 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1459/2879] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:30:57.318535 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:57.329074 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005dc0b-412e-4650-afb9-9551f61f68a7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:57.334243 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:57.335137 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:57.335413 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:57.335714 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] PATH_INFO: `/v3/projects/1a3e43fff5a547d7b1881434d9c8fce3/users/4561fc18fd8c428b84ef497d2b7437ba/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:57.352028 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3e43fff5a547d7b1881434d9c8fce3, user_id=4561fc18fd8c428b84ef497d2b7437ba, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:57.354710 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:57.435170 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1421/2880] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:30:57 2022] PUT /identity/v3/projects/1a3e43fff5a547d7b1881434d9c8fce3/users/4561fc18fd8c428b84ef497d2b7437ba/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:30:57.439213 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:57.449895 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7c156-027e-48ea-8c5b-dcd9b5bd185c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:57.507586 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4591a92-7a64-49f3-8c89-1b243a1621f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:57.508326 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4591a92-7a64-49f3-8c89-1b243a1621f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:57.508611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4591a92-7a64-49f3-8c89-1b243a1621f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:57.508888 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4591a92-7a64-49f3-8c89-1b243a1621f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:57.509619 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4591a92-7a64-49f3-8c89-1b243a1621f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:57.511230 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4591a92-7a64-49f3-8c89-1b243a1621f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:57.522707 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1460/2881] 10.222.0.71 () {58 vars in 1121 bytes} [Mon May 16 15:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 15:30:57.526930 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:30:57.538382 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a391759-4f0f-4704-b71d-335b812c976e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:30:57.543086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:30:57.544023 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:30:57.544307 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:30:57.544587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] PATH_INFO: `/v3/projects/1a3e43fff5a547d7b1881434d9c8fce3/users/4561fc18fd8c428b84ef497d2b7437ba/roles/1b3a0717a878438ea224dec0c7639471` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:30:57.546587 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3e43fff5a547d7b1881434d9c8fce3, user_id=4561fc18fd8c428b84ef497d2b7437ba, role_id=1b3a0717a878438ea224dec0c7639471)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:30:57.549869 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:30:57.560913 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1422/2882] 10.222.0.71 () {60 vars in 1482 bytes} [Mon May 16 15:30:57 2022] PUT /identity/v3/projects/1a3e43fff5a547d7b1881434d9c8fce3/users/4561fc18fd8c428b84ef497d2b7437ba/roles/1b3a0717a878438ea224dec0c7639471 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 15:31:02.948023 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d783007-4578-43f0-8003-677987c0eb7c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:31:02.948523 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d783007-4578-43f0-8003-677987c0eb7c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:31:02.948636 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d783007-4578-43f0-8003-677987c0eb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:31:02.995177 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8d783007-4578-43f0-8003-677987c0eb7c None None] MFA Rules not processed for user `4561fc18fd8c428b84ef497d2b7437ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:31:02.996429 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d783007-4578-43f0-8003-677987c0eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:31:03.073854 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1461/2883] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:31:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:31:03.094647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:31:03.094647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:31:03.169166 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41993-0434-4b45-b7f9-651810caaeaf admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:31:03.175057 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:31:03.175757 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:31:03.176046 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:31:03.176330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:31:03.234346 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:31:03.236067 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:31:03.245891 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1423/2884] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:31:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:31:21.247275 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a44fb-9c12-493f-85c0-6fc56e86ce23 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:31:21.247835 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a44fb-9c12-493f-85c0-6fc56e86ce23 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:31:21.247944 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a44fb-9c12-493f-85c0-6fc56e86ce23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:31:21.265336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b21a44fb-9c12-493f-85c0-6fc56e86ce23 None None] MFA Rules not processed for user `1f3e4b15a6cc4879b24d7a44669a3690`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:31:21.266513 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b21a44fb-9c12-493f-85c0-6fc56e86ce23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:31:21.273855 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1462/2885] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:31:21.575804 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:31:21.577613 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:31:21.634638 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80279020-0c12-436f-a61d-d4e835a194af service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:31:21.638210 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:31:21.638853 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:31:21.639135 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:31:21.639409 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:31:21.641178 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:31:21.650726 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:31:21.652173 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:31:21.660201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1424/2886] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:31:30.278231 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a44fb-9c12-493f-85c0-6fc56e86ce23 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:31:30.289544 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a44fb-9c12-493f-85c0-6fc56e86ce23 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:31:30.293335 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:31:30.293997 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:31:30.294283 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:31:30.294560 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:31:30.296435 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:31:30.306882 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:31:30.308363 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:31:30.316482 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1463/2887] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:31:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:31:45.308320 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:31:45.316740 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59c11e-0b75-41fe-9ca7-1849563aa03a service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:31:45.320964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:31:45.321599 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:31:45.321882 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:31:45.322159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:31:45.330769 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:31:45.332221 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:31:45.341616 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1425/2888] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:31:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:31:47.288290 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:31:47.300923 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519c3dc-42ca-4dc4-bf55-181ac59b34b7 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:31:47.307066 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ffd1f-d254-487e-8f46-c4f70285ec5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:31:47.307777 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ffd1f-d254-487e-8f46-c4f70285ec5e service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:31:47.308046 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ffd1f-d254-487e-8f46-c4f70285ec5e service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:31:47.308321 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ffd1f-d254-487e-8f46-c4f70285ec5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:31:47.310103 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b09ffd1f-d254-487e-8f46-c4f70285ec5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:31:47.322255 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ffd1f-d254-487e-8f46-c4f70285ec5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:31:47.323864 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ffd1f-d254-487e-8f46-c4f70285ec5e service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:31:47.334395 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1464/2889] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:33:44.364495 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:33:44.365910 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:33:44.382215 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cfef4-9435-4fea-80d6-2d1857cd7e6f service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:33:44.387680 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:33:44.388330 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:33:44.388615 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:33:44.388894 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:33:44.390813 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:33:44.409939 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:33:44.411744 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:33:44.428970 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1426/2890] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:33:46.205770 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4957cd-46a7-411a-9094-ef27d823dab0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:33:46.206699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4957cd-46a7-411a-9094-ef27d823dab0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:33:46.206699 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4957cd-46a7-411a-9094-ef27d823dab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:33:46.223038 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4d4957cd-46a7-411a-9094-ef27d823dab0 None None] MFA Rules not processed for user `1f3e4b15a6cc4879b24d7a44669a3690`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:33:46.224247 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4d4957cd-46a7-411a-9094-ef27d823dab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:33:46.231305 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1465/2891] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:33:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:33:47.200765 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:33:47.202250 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:33:47.402167 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcd9df-26b3-4d32-b8c4-242bc0cf6ed4 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:33:47.406981 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:33:47.407618 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:33:47.407927 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:33:47.408201 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:33:47.410963 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:33:47.421932 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:33:47.423498 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:33:47.434024 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1427/2892] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:34:00.992680 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4957cd-46a7-411a-9094-ef27d823dab0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:34:01.006314 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4957cd-46a7-411a-9094-ef27d823dab0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:34:01.012111 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3890e-32c1-45dd-9153-ec4941c70ab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:34:01.017444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3890e-32c1-45dd-9153-ec4941c70ab0 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:34:01.017444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3890e-32c1-45dd-9153-ec4941c70ab0 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:34:01.017444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3890e-32c1-45dd-9153-ec4941c70ab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:34:01.017444 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c3b3890e-32c1-45dd-9153-ec4941c70ab0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:34:01.028739 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3890e-32c1-45dd-9153-ec4941c70ab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:34:01.030330 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3890e-32c1-45dd-9153-ec4941c70ab0 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:34:01.045794 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1466/2893] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:34:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:34:24.354708 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:34:24.365738 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362387-5a64-493f-afc0-b43660017965 service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:34:24.370797 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:34:24.371450 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:34:24.371756 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:34:24.372026 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:34:24.373748 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:34:24.385481 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:34:24.387022 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:34:24.397090 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1428/2894] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:34:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:35:52.872249 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3852cc9-7ff2-4316-8980-a9452af2aa13 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:35:52.872814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3852cc9-7ff2-4316-8980-a9452af2aa13 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:35:52.872814 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3852cc9-7ff2-4316-8980-a9452af2aa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:35:52.891058 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e3852cc9-7ff2-4316-8980-a9452af2aa13 None None] MFA Rules not processed for user `1f3e4b15a6cc4879b24d7a44669a3690`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 15:35:52.892556 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e3852cc9-7ff2-4316-8980-a9452af2aa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:35:52.899508 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1467/2895] 10.222.0.71 () {58 vars in 957 bytes} [Mon May 16 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 15:35:52.934953 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:35:52.937004 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:35:53.019644 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d4daf-3164-47a9-8a75-78319369c306 service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:35:53.023259 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f2421c1bfb5402da3179e6f3c749374', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:35:53.023866 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:35:53.024151 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:35:53.024424 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:35:53.026154 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:35:53.039044 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:35:53.040486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:35:53.051226 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1429/2896] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:35:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:35:53.324742 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3852cc9-7ff2-4316-8980-a9452af2aa13 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:35:53.335772 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3852cc9-7ff2-4316-8980-a9452af2aa13 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:35:53.339197 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:35:53.339892 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:35:53.340172 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:35:53.340445 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:35:53.342949 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:35:53.352373 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:35:53.353832 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:35:53.361792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1468/2897] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:35:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 15:35:54.103789 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:35:54.113352 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3478a0-5c39-40c9-b915-e6e83621b431 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:35:54.116952 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:35:54.117586 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:35:54.117868 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:35:54.118146 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:35:54.119933 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:35:54.133935 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:35:54.135411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:35:54.146762 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1430/2898] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:35:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:36:03.296383 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:36:03.309563 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27001486-16b6-4e3b-bfd3-e9c1700078b0 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:36:03.314953 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:36:03.315685 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:36:03.315969 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:36:03.316242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:36:03.323870 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:36:03.325483 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:36:03.334712 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1469/2899] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:36:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:37:27.096300 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:37:27.131257 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b82e5c-7579-4ff5-930d-fcb316601ee9 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:37:27.134862 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:37:27.135489 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:37:27.135794 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:37:27.136065 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:37:27.143397 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:37:27.144912 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:37:27.153086 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1431/2900] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:37:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:37:42.618580 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:37:42.663336 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ca4b-6c3a-4906-b845-8b58a1305243 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:37:42.670207 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d58a6e2eac047ad8e88a1f973ecc697', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:37:42.670881 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:37:42.671165 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:37:42.671442 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:37:42.681800 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:37:42.683593 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:37:42.696494 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1470/2901] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:37:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:37:56.941455 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:37:56.950350 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ddcb-a26d-40e5-ad03-b9fd97d26d9b service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:37:56.954014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b2a2d381a14f5fb04e721905b36ace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:37:56.954647 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:37:56.954942 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:37:56.955209 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:37:57.013302 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:37:57.014725 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:37:57.024297 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1432/2902] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:37:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 15:38:03.757180 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:38:03.769185 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788341a-aa28-4299-b747-b30500d66c0c service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:38:03.774792 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:38:03.775698 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:38:03.775988 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:38:03.776269 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] PATH_INFO: `/v3/users/1f3e4b15a6cc4879b24d7a44669a3690` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:38:03.777475 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3e4b15a6cc4879b24d7a44669a3690)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:38:03.779502 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:38:03.923732 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3e4b15a6cc4879b24d7a44669a3690'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:03.973611 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3e4b15a6cc4879b24d7a44669a3690'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:03.981803 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3e4b15a6cc4879b24d7a44669a3690'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:03.982506 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3e4b15a6cc4879b24d7a44669a3690'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:03.993903 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3e4b15a6cc4879b24d7a44669a3690'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:03.999665 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1471/2903] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:38:03 2022] DELETE /identity/v3/users/1f3e4b15a6cc4879b24d7a44669a3690 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:38:04.238720 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:38:04.249844 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0652fe-97b5-4d90-9824-cc4d6e9a0ad6 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:38:04.306114 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:38:04.306964 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:38:04.307242 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:38:04.307511 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] PATH_INFO: `/v3/projects/35be16de8a104d569efbb5633c5036af` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:38:04.308689 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35be16de8a104d569efbb5633c5036af)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:38:04.310568 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:38:04.418793 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35be16de8a104d569efbb5633c5036af', 'request_id': 'req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.479920 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35be16de8a104d569efbb5633c5036af', 'request_id': 'req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.485351 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35be16de8a104d569efbb5633c5036af', 'request_id': 'req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.496244 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] The token cache is being invalidate because project 35be16de8a104d569efbb5633c5036af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:38:04.498356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.498356 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.498704 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1433/2904] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:38:04 2022] DELETE /identity/v3/projects/35be16de8a104d569efbb5633c5036af => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:38:04.502703 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:38:04.504121 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:38:04.527505 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036114d2-561a-4dda-bab8-592ea47a5d7d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:38:04.583163 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:38:04.584166 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:38:04.584452 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:38:04.584776 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] PATH_INFO: `/v3/users/4561fc18fd8c428b84ef497d2b7437ba` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:38:04.586386 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4561fc18fd8c428b84ef497d2b7437ba)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:38:04.587977 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:38:04.769978 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4561fc18fd8c428b84ef497d2b7437ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.848042 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4561fc18fd8c428b84ef497d2b7437ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.855684 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4561fc18fd8c428b84ef497d2b7437ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.856242 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4561fc18fd8c428b84ef497d2b7437ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.868067 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4561fc18fd8c428b84ef497d2b7437ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:04.874292 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1472/2905] 10.222.0.71 () {58 vars in 1223 bytes} [Mon May 16 15:38:04 2022] DELETE /identity/v3/users/4561fc18fd8c428b84ef497d2b7437ba => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:38:05.101774 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:38:05.103188 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:38:05.115851 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a6708-eb8c-40d3-a0fa-0e6d6e10703d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:38:05.173861 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6246a26f46e94e7389a45ee846fec1c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63e6523cee6045dabb771a1e00b70243', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:38:05.174722 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:38:05.175005 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:38:05.175283 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] PATH_INFO: `/v3/projects/1a3e43fff5a547d7b1881434d9c8fce3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:38:05.176486 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a3e43fff5a547d7b1881434d9c8fce3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:38:05.178411 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:38:05.297717 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a3e43fff5a547d7b1881434d9c8fce3', 'request_id': 'req-c749fa79-e6ff-43a8-989d-d4f2c4f474db'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:05.368014 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a3e43fff5a547d7b1881434d9c8fce3', 'request_id': 'req-c749fa79-e6ff-43a8-989d-d4f2c4f474db'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:05.373678 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a3e43fff5a547d7b1881434d9c8fce3', 'request_id': 'req-c749fa79-e6ff-43a8-989d-d4f2c4f474db'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:05.385183 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] The token cache is being invalidate because project 1a3e43fff5a547d7b1881434d9c8fce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 15:38:05.385463 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:05.386159 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-c749fa79-e6ff-43a8-989d-d4f2c4f474db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 15:38:05.387850 dv-cinder-841753-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1434/2906] 10.222.0.71 () {58 vars in 1232 bytes} [Mon May 16 15:38:05 2022] DELETE /identity/v3/projects/1a3e43fff5a547d7b1881434d9c8fce3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 15:39:30.391088 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 15:39:30.392827 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:39:30.480596 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d122f7c-e98c-46b4-aaa4-787c58c3f74b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 15:39:30.538352 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33af22-a0de-4f93-ab30-455fa1e998b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c36906c508f41cca018e5f6dcaadd91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ef95ef9b6a74897922ac44c55f68e3b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 15:39:30.539001 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33af22-a0de-4f93-ab30-455fa1e998b2 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 15:39:30.539284 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33af22-a0de-4f93-ab30-455fa1e998b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 15:39:30.539568 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33af22-a0de-4f93-ab30-455fa1e998b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 15:39:30.541353 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d33af22-a0de-4f93-ab30-455fa1e998b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 15:39:30.552908 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33af22-a0de-4f93-ab30-455fa1e998b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 15:39:30.554488 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33af22-a0de-4f93-ab30-455fa1e998b2 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 15:39:30.565076 dv-cinder-841753-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1473/2907] 10.222.0.71 () {60 vars in 1365 bytes} [Mon May 16 15:39:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)